[HN Gopher] Google DeepMind shifts from research lab to AI produ...
___________________________________________________________________
Google DeepMind shifts from research lab to AI product factory
Author : kjhughes
Score : 200 points
Date : 2024-06-17 22:02 UTC (1 days ago)
(HTM) web link (www.bloomberg.com)
(TXT) w3m dump (www.bloomberg.com)
| l1n wrote:
| https://archive.is/5XdU8
| jsemrau wrote:
| "The Overviews launch didn't go well." understatement of the
| year.
| kevindamm wrote:
| One thing that classic Google did right was embed researchers
| into product groups. It's true there were always some teams that
| were pure research but for the most part researchers were working
| within a product group.
|
| Then some acquisitions and internal musical chairs and it became
| less like that. Now I'm not all doom and gloom like this article
| (although with DeepMind why not leave well enough alone? They do
| excellent research). But, it does seem suboptimal to pivot all
| the way to AI Product Factory... were there no other existing
| product factories they could have turned instead?
| curious_cat_163 wrote:
| I agree: hybrid teams with a diversity of product/research
| skills at the team level is the way to go. It is thinkers and
| doers that need to come together.
|
| It is way easier said than done, though. You need true buy in
| from a ton of stakeholders -- employees being the primary ones.
| And people get set in their ways.
|
| I do like a product bias though. Not because it is more
| valuable somehow but because it provides the applied scientists
| deeper exposure to the problem space, early and often.
| flakiness wrote:
| The heads of the research once wrote about that. They called it
| a "Hybrid Approach to Research" (as other comments pointed
| out).
|
| https://static.googleusercontent.com/media/research.google.c...
| moandcompany wrote:
| The "Hybrid Approach to Research" paper describes how "Google
| Research" first started when Google was mostly, if not
| entirely, about Search and "Research" was part of the Search
| organization.
|
| In these days, there were no "pure research" roles, nor were
| there formal designations for "research scientists" as a
| career ladder at Google. There were "SWEs" and in some cases
| "Members of Technical Staff."
|
| Since then, "Research" became its own organization or
| "Product Area" at Google (i.e. the equivalent of a company
| division). "Google Brain" was also created. Deepmind was
| acquired. All of these existed simultaneously, however
| Deepmind remained as an organizationally separate entity. In
| this era, the "Research Scientist" role was created, which
| generally existed exclusively within "Google Research." A
| large span of this era had John Giannandrea ("JG") at the
| helm of the Google Research org; (note: Giannandrea left to
| head and build Apple's "AI/ML" organization, which includes
| Siri, a few years ago.
|
| After JG's departure, Google Brain and Google Research were
| brought together under the common leadership of Jeff Dean, as
| an organization called still called "Google Research" with a
| branch still called "Google Brain." For perspective, it may
| be useful to consider too that the size of "Google Research"
| in staff headcount here measured in the several-thousands.
| This configuration existed for the last few years, with the
| latest changes being the merging of "Google Research" and
| Deepmind into "Google Deepmind."
|
| -
|
| I am a Xoogler, formerly from this product area. One of the
| things I and at least a few others observed was that
| "Research" was becoming defacto synonymous with "Machine
| Learning / AI," yet not all of Google's storied research
| accomplishments, or problem areas, are limited to Machine
| Learning and AI.
|
| In the last few years, Google made its public statements of
| being an "AI-first," previously "mobile-first," company in
| recognition that it would be incorporating and leveraging ML
| and AI technology across all of its products and services.
|
| This raised a significant question: What should "Google
| Research" or Research at Google be if product areas across
| Google began full incorporation of AI/ML technology and
| methods in their products? What if they incorporated their
| own AI/ML teams? If Google was truly successful at becoming
| AI-first, how should "Google Research" define and focus its
| organizational purpose, research portfolio, and show its
| value when Moonshots/X also exists within Alphabet? Over
| time, there were many parts of "Google Research" and research
| at large across Alphabet that felt that their purpose, or at
| least their individual reason for joining, was to do "pure
| research," yet this is not how the organizations started at
| all in the beginning. Many researchers and teams also knew
| that for practical reasons (e.g. promotion) that they
| generally needed to present and align their work with things
| like product launches with partner organizations.
|
| I suppose we are seeing some of the answer to this with
| Google DeepMind stating that they will be aligning more
| strongly with creating AI products, but in addition to the
| question of what happens to foundational research (for AI),
| what happens to foundational research in non-AI areas for
| Google and Alphabet?
| radicaldreamer wrote:
| Why? Attach seasoned product teams from Google to the research
| org and have experienced PMs paired with experienced EMs launch
| products.
|
| Trying to transition a research org into a product org is going
| to be needlessly painful, especially since the research org needs
| to be firing on all cylinders in this hyper-competitive space.
| advael wrote:
| God, yea. I've flipped between R&D and product development a
| few times in my career, sometimes at the same company, and it's
| a really rough transition even for an individual experienced in
| both. I can't imagine trying to flip a whole research team to
| make products is going to go well, especially when the products
| are getting a ton of well-deserved bad press and a lot of those
| researchers were coming from academia rather than elsewhere in
| industry in the first place
| gaogao wrote:
| Agreed. As an example of that, Meta's kept its research org,
| FAIR, still doing fundamental research. Research orgs are great
| at demos, but actual productionalization takes a different
| mindset.
| nomad_horse wrote:
| FAIR is considerably downscaled from what it was before, in
| eg 2022.
| fooker wrote:
| Fun anecdote: some folks from FAIR reached out to a friend,
| asked her to apply, and then rejected her without an
| interview!
| htrp wrote:
| Facebook's recruiting funnel is super broken after they
| fired a ton of their HR teams
| oivey wrote:
| More than likely for the sorts of products they want to make
| you still need very deep research expertise that random product
| teams won't have.
| lupire wrote:
| Deep research expertise into tuning an LLM into a user
| friendly product?
|
| Who do you think has that expertise? The people working on
| the model or the people studying users?
| 4death4 wrote:
| Definitely the people working on the model. It ultimately
| doesn't matter what the users want because you can't
| arbitrarily deliver an experience. You can only deliver
| what it's possible to extract from the model, so growing
| the possible things the model can do well is most
| important.
| ethbr1 wrote:
| Engineering and product are both important.
|
| Without engineering, you don't have the capability.
|
| Without product, you don't build something users are
| actually interested in.
|
| I've seen too many engineering teams try to productize
| what they want, not what people not-them want, and then
| be flummoxed by lack of adoption.
|
| Nothing sucks more than burning the midnight oil to nail
| a target... that ended up being 2m to the right of the
| actual target.
| rusticpenn wrote:
| I have experience in both R&D and product. Both need
| different approaches to work. The goals of people working
| on the model will be different from product people. As
| mentioned by the other user, a product team can look at
| things produced from research and see how it can bring it
| to users.
| hobs wrote:
| In my mind you could not be proving that we need product
| people more. They'd never say "It ultimately doesn't
| matter what the users want" - they'd say "let's find a
| way to build what users want" not "let's grow the
| possible things a model can do well".
| geodel wrote:
| If engineering/research is all mattered we would have
| maybe two order of magnitude more successful products or
| companies. Because product-market fit is a thing we don't
| have any successful research turning to successful
| product.
| lupire wrote:
| The researchers aren't suddenly building products.
| Vt71fcAqt7 wrote:
| >seasoned product teams from Google
|
| .
| TaylorAlexander wrote:
| Yep. I joined Google X Robotics (which became Everyday Robots,
| which got canceled) just as the org was winding down a big R&D
| push and moving to product development. Engineers were palpably
| hurt by their various projects being canceled, and in my
| opinion they never had a viable product strategy beyond "let's
| see if we can find a consumer use for this robot". This
| strategy ultimately failed and they canceled the project, let
| go a bunch of the people, and now the robots are being used for
| AI research. So the whole shift from R&D to product development
| was a failure. They could have saved a lot of grief and money
| if they just continued as an R&D org, and in my opinion they
| would have left open some important doors which would have
| really helped with AI research.
| astromaniak wrote:
| I'm afraid this is going to be another Everyday Robots or
| Boston Dynamics. Google is ruled by managers, not
| visionaries. Strategy is 'try and see if it works in 4 years.
| if not cancel'. They followed it in many cases. So,
| DeepMind's cancellation is long overdue. A couple of years
| back one of Google's top managers talking about AGI said it's
| most likely to happen in DM. But current LLM boom happened
| elsewhere. Likely managers are disappointed in DM.
| aborsy wrote:
| To fair, Google invented transformers and contributed a lot
| of basic research. OpenAI made it bigger.
| Rinzler89 wrote:
| And Xerox invented the GUI and Kodak the digital camera,
| and look at them today.
|
| Inventing things is no guarantee of success.
| robertlagrant wrote:
| Yes, but look at the previous comment. How does
| "inventing transformers" marry with "Google has managers
| not visionaries"?
| jsjohnst wrote:
| Just because a company is 99% one thing, doesn't mean
| that the 1% remaining don't have moments of real genius.
| What it does mean however is that when those real genius
| moments happen, the company isn't in a good position to
| capitalize on it (ala Xerox or Kodak mentioned
| previously, but there are so many more).
| ToucanLoucan wrote:
| Because that's the hallmark of business-minded people
| being at the helm and not engineers, which has been an
| issue for Google for a long time: the greatest inventions
| ever seen will be squandered in terms of the org itself
| because management can't see beyond the next few quarters
| and won't invest properly in it.
| FactKnower69 wrote:
| ??? Xerox and Kodak are immortal household names that
| dominated their niches for decades, what level of
| "success" would satisfy you? what are you lamenting, that
| they didn't completely enshittify their products while
| they were on top and torch their brands to the ground for
| a little more revenue?
| jsemrau wrote:
| Transformers != Modern LLMs. It is 100% an important
| invention, but other important tech like hidden markov
| chains existed long before.
| woodson wrote:
| Well, strictly speaking modern LLMs are transformers (not
| counting state space models like Mamba, for the moment).
| Not sure what hidden Markov chains have to do with modern
| LLMs.
| TaylorAlexander wrote:
| Well, LLMs are not AGI. They have serious limitations [1]
| and honestly my fear is that it's too soon. I agree Google
| is ruled by managers (that's why I hated it), and my fear
| is that the managers have FOMO and want to push to
| productize asap even tho the tech isn't ready yet.
|
| Look what happened when Google tried to throw an LLM in to
| search. Absolute shitshow. That's not ready to become any
| kind of product!
|
| If they kill R&D now to focus on productizing something
| that is half baked, they will fail to develop those new
| inventions which might get us to AGI. When I worked at
| Google X Robotics I was hired on to the remnants of the
| last research team, which was dissolved six months after I
| started (I was moved to hardware test engineer). Our
| subteam really wanted to research multi-finger grippers but
| we got overruled, so the robot had to do everything with a
| two finger pinch gripper. Which is fine for research but
| absolutely unsuitable for real world tasks. It couldn't
| even operate a spray bottle without special attachments and
| they thought it was going to clean people's homes!
|
| [1] I am sharing this one a lot lately but I'm very moved
| by Yann LeCun's arguments about the limits of
| autoregressive approaches here. As a robotics engineer I
| have been dismayed at all the attention LLMs are getting
| despite serious limitations that make them generally
| unsuitable to solve some of the most important problems in
| robotics. https://youtu.be/1lHFUR-yD6I
| ignoramous wrote:
| btw, Sustkever is on record that Transformers are enough
| to achieve AGI:
| https://www.youtube.com/watch?v=kW0SLbtGMcg&t=33s
| TaylorAlexander wrote:
| Okay. Well he's not omniscient. He gives a pretty weak
| argument by analogy, while LeCun makes multiple detailed
| arguments which I find compelling.
| ignoramous wrote:
| > weak analogy
|
| tbf, Ilya was on a VC podcast not at a tech conf.
|
| > Well he's not omniscient.
|
| Neither is Yann (who has since proposed a different
| architecture / vision which is yet to take off), but my
| comment was meant to highlight a _recent_ claim from
| another accomplished researcher in the field.
| robertlagrant wrote:
| > my fear is that the managers have FOMO and want to push
| to productize asap even tho the tech isn't ready yet
|
| This is exactly it. With the limitations ChatGPT is
| encountering around safety and hallucination, Google
| probably should've just said "we're working on something
| awesome - hold on" and kept plugging away before
| releasing, instead of ex-Product CEO making them release
| something now, even if half of the demo video is fake.
| KoolKat23 wrote:
| But stonks don't go up then. These days stock price bumps
| are more lucrative to shareholders than actual company
| returns.
| robertlagrant wrote:
| But have stonks gone up?
| KoolKat23 wrote:
| I'm sure it's bursting right now but:
|
| "Companies that mentioned AI in earnings saw their stocks
| rise 4.6% on average, a study from Wall Street Zen
| found."
|
| https://markets.businessinsider.com/news/stocks/ai-stock-
| mar...
| golergka wrote:
| I'm using chatGPT to Google something for around a year
| (or whenever bing browsing became available), and I'm yet
| to set a single hallucination based on web search
| results. May be Google is just not very good at this.
| freilanzer wrote:
| > I'm using chatGPT to Google something for around a year
|
| Maybe I'm too tired, but I don't understand this
| sentence.
| jazzyjackson wrote:
| he's been bingin' since last summer, daddy-o
| xdavidliu wrote:
| They have been using ChatGPT to perform all searches for
| a year, where in the past they would have used Google.
| dylan604 wrote:
| Isn't this the mindset that has PoCs released to production?
| boyka wrote:
| Likely that some McKinsey type consultants or ex-consultants in
| senior management deem this to be absolutely necessary and the
| only way to go.
| stephen_cagle wrote:
| I tend to agree. I'm curious whether this is Deepmind saying "I
| think we could do things better, let's do this ourselves" or
| the leadership of Alphabet saying "Get these ivy league
| intellectuals to prioritize productionizing products!"
|
| Would seem far more sensible to allow Deepmind to continue to
| release hit after hit in the ML research world, and simply
| embed "fly on the wall" PM's into their org that can
| independently productionize any golden nuggets they happen to
| create.
| TeMPOraL wrote:
| I feel it's more of leadership of Alphabet saying, "Our
| people literally _invented transformers_ , so how come we're
| at the bottom of AI race instead of at the top? A random non-
| profit took out research and run with it, and they're the
| hottest company in the world now. This is unacceptable![0]".
|
| Bad idea. People good or lucky enough to land in R&D like
| doing R&D. Force them to be product people, I expect most of
| them will leave.
|
| --
|
| [0] - Like Muffin from Bluey,
| https://youtu.be/hZVlBQXVtZA?t=8.
| smallnamespace wrote:
| Yes, but that doesn't explain how OpenAI able to walk and
| chew bubble gum at the same time, unless you think there is
| some extra spark that Google is lacking.
| TeMPOraL wrote:
| I think OpenAI was chilled out, and mostly lucky. Chilled
| in the sense that Google seems to be too full of itself,
| and their practice of publishing groundbreaking research
| without actually publishing anything other people can use
| is, frankly, annoying. OpenAI managed to leapfrog them by
| slapping a chat interface on a tuned GPT-3 and putting it
| on the Internet.
|
| The chat service bit is them being chill, but the real
| spark was the model. I say they were lucky, because AFAIK
| back then no one expected LLMs to show so many and so
| advanced general capabilities. This took everyone by
| surprise, and since people could already play with it,
| ChatGPT took off on its own - it had so much real,
| transformative value, that it spread out with zero
| marketing. That's a rare, bona fide case of "word of
| mouth", it was just _that useful_. But that wasn 't a
| strategy, that was luck.
|
| To their credit though, OpenAI turned this early win into
| an opportunity and is excellent at exploiting it. Being
| small helps.
| cma wrote:
| GPT-3 was out for years already for Googlw to see, but
| with what OpenAI saw of it they began training an
| expensive GPT-4, before chatgpt success, maybe started
| before RLHF.
| ethbr1 wrote:
| > _it had so much real, transformative value, that it
| spread out with zero marketing_
|
| IMHO, in retrospect, the failure gradient of early LLMs
| is underappreciated in driving adoption.
|
| Windows 95 failure: blue screen with inscrutable error
| code. Everyone noticed that.
|
| LLM failure: run-around non-answer (user shrugs and tries
| again) or confident and plausible incorrect answer (user
| doesn't recognize this without research).
|
| Essentially, the ways in which LLMs didn't work were the
| most hidden and hardest to discover failure mode.
|
| Which was perfectly tuned for the "I'm going to try this
| thing for 5 minutes and be amazed" first impression.
|
| Which allowed subsequent generations to backfill the
| capability gaps.
|
| Tl;dr - We shouldn't underappreciate quiet-failing as a
| product adoption driver.
| FactKnower69 wrote:
| >We shouldn't underappreciate quiet-failing as a product
| adoption driver.
|
| I'm certain it drives a lot of early user retention in
| the short term, but I feel strongly that this is
| ultimately a very myopic view which will prove
| catastrophic in the long term in much the same way that
| swallowing exceptions at runtime builds compounding
| technical debt you'll have to reckon with sooner or later
|
| more broadly, there is just so much handwaving away all
| the black box parts of deep neural networks that are
| completely opaque and there seems to be very little
| interest in building the tooling to properly visualize,
| explore, and DEBUG latent space; until those priorities
| change this whole thing is a huge time bomb.
|
| imagine if instead of coming with full memory dumps and
| diagnostic codes, BSODs just said "sorry, your computer
| had an oopsie!", and not a single engineer at Microsoft
| had a complete understanding of _why_ the BSOD happened
| in the first place; sometimes it just does that! whoops!
| ethbr1 wrote:
| > _imagine if instead of coming with full memory dumps
| and diagnostic codes, BSODs just said "sorry, your
| computer had an oopsie!"_
|
| So, MacOS? ;)
|
| In all seriousness, I wasn't opining on the usefulness of
| opaque/hidden errors, but rather the effectiveness of
| them.
|
| In an alternate reality where the first LLMs instead spit
| back an error reference instead of English, I don't think
| we would have seen nearly as rapid mass market adoption.
|
| And, not to put too fine a point on it, early
| conversational LLMs and image diffusion models were
| literally trained so their junk output is as plausible as
| possible.
| KoolKat23 wrote:
| I agree it is this,
|
| I also think people and society also give themselves way
| too much credit for their successes. There's plenty of
| smart hardworking people out there who continue to
| contribute but never stumble upon a unicorn. To a large
| extent its luck, a much larger contributor than people
| realize. All you can do is to play the game, consistently
| contribute and work hard on R&D and products and you
| improve your odds of stumbling upon success. But it's never
| guaranteed.
| hotstickyballs wrote:
| That would require Google to have enough "seasoned product
| teams"
| michaelt wrote:
| Google has cancelled more products that most companies have
| ever launched.
|
| You'd think they'd have plenty of spare people with product
| launch experience.
| hot_gril wrote:
| Maybe the only people they retained have product turndown
| experience.
| lenerdenator wrote:
| > Why?
|
| Must transfer value, and the guy in charge of the company is
| not good at allocating the company's resources to do that with
| an eye on long-term results.
| pixiemaster wrote:
| well actually i don't think google has good PMs - besides
| adwords and android, there are no really successful products.
| whywhywhywhy wrote:
| > Why?
|
| Because the current way they were working squandered over a
| decade lead in the space. Deep Dream was 2015... Google Magenta
| was 2017...
| hot_gril wrote:
| Maybe that's what they're doing. The article just says they're
| combining the two labs, not much further detail.
| curious_cat_163 wrote:
| > While no one is getting as much computing power as they want,
| the supply is tighter for teams engaged in pure research, say the
| former employee and others familiar with the lab.
|
| Good! Maybe they will focus on researching how to make these
| things more compute efficient.
| caycep wrote:
| true. I wonder how much energy in food/farming to develop a 6
| yr old human is required vs. the amount required to run a
| hojillion GPUs running the latest generative algorithms
| dontlikeyoueith wrote:
| Assuming dollar cost is a relevant metric, the 6yr old human
| is far cheaper.
| whamlastxmas wrote:
| The training costs of the 6 year old are millions of years of
| evolution and suffering of billions of people.
| lupire wrote:
| Irrelevant, as that's already been paid.
| FactKnower69 wrote:
| sunk cost fallacy
| Izikiel43 wrote:
| But can a 6 year old answer millions upon millions of
| queries?
|
| You need a lot of 6 year olds.
| DrScientist wrote:
| Hasn't everything in ChatGPT - been posted by 6 year olds
| on reddit, and ChatGPT is simply a very impressive indexer
| and query interface? :-)
|
| The real question is ChatGPT a better information retrieval
| tool that the old Google search interface before they
| dumbed it down?
|
| For me the main differences are that for ChatGPT it
| summarises across multiple sources - sometimes good,
| sometimes not, and the refinement of queries feels much
| more natural with it's use of context.
|
| Though I often find myself fighting both the new Google
| search interface and ChatGPT to try and get the right
| answers to the specific area I want.
| pie420 wrote:
| luckily, we have hundreds of millions of 6 year olds
| addicted to reddit who will answer whatever question you
| might have.
| ericd wrote:
| Most people can probably answer about some subset of
| questions better than GPT4 can, but I don't think there's a
| human alive who could answer nearly as competently on >50% of
| the questions if gets asked. So I don't know why you'd
| benchmark it against a 6 year old. If you compare the carbon
| impact of training one of these to the carbon footprint of
| the average American family, it's an _incredible_ deal in
| terms of utility.
| dinobones wrote:
| Overall probably a good move for Google. They were were spending
| 100s of millions of dollars a year to train RL agents to get good
| at playing Qwop for close to a decade.
| 29athrowaway wrote:
| Or chess, go, shogi, arimaa, StarCraft II, etc.
| summerlight wrote:
| In a PR perspective, at least AlphaGo did a prominent job for
| Google. It failed to keep its prestigious status but at least
| it's a leadership/product side issue, not DeepMind's.
| hot_gril wrote:
| That stuff gave Google the appearance of "nerds playing
| with their toys." Sure it's worthwhile research, but it's
| not great how many things Google announced that close to 0
| people could use. They're trying to be more user-oriented
| now, and because of Cloud, their users are businesses.
| FactKnower69 wrote:
| No. AlphaGo et al gave the appearance of "world class
| researchers achieving state of the art on problem classes
| that humans have been devoting lifetimes of study for
| thousands of years". the current desperate LLM scramble,
| including stuffing shitty, hallucinating, half-baked
| paraphrase spam to the top of every Google search result
| page, gives the appearance of "panicked flailing and
| tacit admission that high level decisionmaking has been
| captured by terrified MBA types"
| hot_gril wrote:
| There's a good long middle ground. Search results do seem
| desperate now, but they got into this position in the
| first place by failing to execute when they had the
| advantage. None of Google's customers have ever cared how
| well an AI can play Go. If there's one place the bragging
| could've translated to profits, it'd be Tensorflow
| dominance, which didn't happen.
| aprilthird2021 wrote:
| I strongly disagree. Research is how you stay ahead of your
| competitors, yes even silly sounding research like training RL
| agents to play Qwop!
|
| Research is how Google invented the transformer that underlies
| so many current Gen AI models.
|
| Pivoting research to products is exactly the kind of short term
| thinking consultants or mercenaries would propose, get promoted
| off of, and leave just before the consequences start rearing
| their head
| hot_gril wrote:
| They did research for years, but they didn't stay ahead of
| their competitors. They gave transformers away for free.
| FactKnower69 wrote:
| RNN gameplay agents and reinforcement learning are a thousand
| times more interesting than stochastic parrots that rephrase
| Google for you, you'll see once this hype wave finishes dying
| out in a couple years
| light_triad wrote:
| ___
| HWR_14 wrote:
| What's the issue with large companies plowing their profits
| into pure research?
| viscanti wrote:
| It seems to be difficult to turn the pure research back into
| new products. Apple famously got lots of ideas for free from
| Xerox PARC. Google researchers wrote the Attention Is All You
| Need paper and they're now desperately playing catchup
| because they couldn't convert it to any kind of product.
| There's nothing wrong with companies investing in pure
| research, but these large companies sometimes are unable to
| take advantage of the research. The people running the
| business want to keep doing what got them successful, not
| some new experimental thing that might not work.
| chasd00 wrote:
| Yeah being first doesn't mean you win automatically.
| There's a story about the Ramones playing a show at a
| famous club in NYC and everyone in the crowd went home and
| started bands that became way more successful and famous
| than who they were trying to be like. ...I think blondie
| was one of the bands that came out of that crowd.
| n4r9 wrote:
| You might be thinking of the Sex Pistols gig whose
| audience of 30-40 included Morrissey, Mark E Smith, the
| Buzzcocks and Lower Broughton: https://www.bbc.co.uk/manc
| hester/content/articles/2006/05/11...
|
| The docudrama 24 hour party people is good to watch about
| this era.
| n4r9 wrote:
| Sorry, mental blip, replace "Lower Broughton" with "Joy
| Division" (!)
| aprilthird2021 wrote:
| But everyone and their mom would rather be the Ramones
| than Blondie, LOL
| aprilthird2021 wrote:
| > Google researchers wrote the Attention Is All You Need
| paper and they're now desperately playing catchup because
| they couldn't convert it to any kind of product.
|
| This isn't true. The transformer underlied Google Translate
| for a long time. They just didn't monetize Google Translate
| heavily enough. It's still one of the best translation
| services out there. And its ability to translate real-time
| conversations has been around for years now.
| light_triad wrote:
| ___
| karmasimida wrote:
| AI product factory seems like an odd choice of phrasing ...
| kevinventullo wrote:
| It's MBA-speak. "So we put in researchers, and products and
| money come out. This slide deck is gonna get me promoted."
| ruraljuror wrote:
| > In May, the lab released a new version of AlphaFold, a landmark
| tool for predicting protein structures. Hassabis says it could
| develop into a $100 billion business, but some people at Google
| have questioned whether he should be dedicating so much time to
| it.
|
| Wonder what the timeframe on that speculative return is?
| prithvi24 wrote:
| > Researchers inside the AI unit have told colleagues they're
| proud of their advances on Gemini, such as its "context window,"
| the amount of information the system can analyze at once. This is
| particularly useful to a company whose enormous amount of data is
| one of its key competitive advantages.
|
| what does a large context window have anything to do with
| google's data moat?
| gradus_ad wrote:
| It's the sort of careless, superficially meaningful statement
| that an LLM would make...
| mitthrowaway2 wrote:
| My reading is that the large context window makes Gemini
| _useful as a tool to Google_.
| ethbr1 wrote:
| > _Hassabis says that he's learning more about introducing
| products and that Google's product teams, in turn, are dealing
| with the novel challenges of generative AI, which has the
| potential to behave unusually when placed in the hands of the
| general public._
|
| This part isn't rocket science.
|
| Step 1) Post on 4chan and SomethingAwful "What is the worst thing
| you could do with genAI? Go."
|
| Step 2) Test your beta product against all the answers you get.
| Barrin92 wrote:
| the tricky part with systems like these isn't to find and fix
| the worst things that people can come up with on 4chan because
| they're by definition obvious. The much trickier part is
| finding the little problems that way more people run into and
| that most people might not even immediately recognize or
| report.
|
| And that is a very complicated science in particular with
| something that can be as fuzzy and intransparent as generative
| AI.
| eitland wrote:
| This has been going on since long before[1] the recent AI
| craze.
|
| IMNSHO it seems Google just cannot miss an opportunity to
| mess up the basics in the quest for amazing and then fail at
| amazing or cancel it just as they are about to achieve it.
| And, ironically this has transformed their search engine from
| unbeatable leader in its field to something much closer to
| what it replaced.
|
| [1]:This is from 5 years ago: https://erik.itland.no/more-
| fun-with-google-mixing-images-fr...
| robertlagrant wrote:
| > the worst things that people can come up with on 4chan
| because they're by definition obvious
|
| If they're finding your flag through star patterns and flying
| drones at it to set it on fire, I think they go a bit deeper
| than "obvious".
| cloudking wrote:
| I think the challenge is you can't QA every edge case, because
| there's unlimited edge cases.
| HeatrayEnjoyer wrote:
| The Control Problem.
| ethbr1 wrote:
| There's also a standard distribution that most people will
| think of.
|
| "Generate photos of Nazis" wouldn't have been my first use,
| but in retrospect it does seem like something that of course
| The Internet is going to try.
|
| That Google didn't even identify that sort of low-hanging
| fruit as a QA case is what points to a process in need of
| external input.
| therobots927 wrote:
| Hadn't these two AI orgs within Google been fighting over
| resources for a long time? At the end of the day that's just
| counterproductive. A merger was all but guaranteed and it's clear
| given current stock market sentiment why the product team was
| chosen. Doesn't mean I don't feel bad for the Deepmind
| researchers impacted. The genAI hype is sparing no one, not even
| the foremost AI labs in the country.
| banish-m4 wrote:
| Meta needs to emulate this because they're sinking tens of
| billions without being focused enough on delivering profits with
| all that hardware they're spending treasure on.
| fhub wrote:
| Meta did $12.4B profit in Q1 2024 which was 116.7% increase
| year-over-year. They have a lot of treasure to spend and the
| treasure chest keeps getting bigger.
| aprilthird2021 wrote:
| Pretty untethered take. Meta has seen one of the largest
| growths in profit in the past few years. Their improvements in
| AI have made far more profitable their AI ads targeting
| business
| nelsonic wrote:
| Let's not forget that Demis Hassabis (DeepMind CEO) created Theme
| Park so he knows how to create products. I have full confidence
| in his leadership. Buy more Alphabet (GOOG) shares!
|
| Ref: https://en.wikipedia.org/wiki/Demis_Hassabis#Bullfrog
| stephen_cagle wrote:
| ...and (burying the lead) wrote it while he was under 18! But,
| that is one of the odder takes for why he would be good at
| building a product?
| nelsonic wrote:
| Demis understands the "customer" and can use everything he
| has learned in the last 20 years to build something
| incredible. If he can build a great/successful game with low
| resources, he will smash a consumer product with unlimited
| resources and excellent people.
| seanhunter wrote:
| I definitely don't think that follows at all.
| pixelpoet wrote:
| Saw him briefly at my first job at Lionhead Studios, and also
| worked with Alex Evans (Media Molecule cofounder, coauthor of
| InstantNGP, legendary demoscener, ...) there. Pretty amazing
| how much talent was buzzing around there.
| nabla9 wrote:
| Hassabis moves from fundamental AI research into app development.
|
| (alternative title)
| chucke1992 wrote:
| Essentially Google has no idea how to leverage their AI as Gemini
| for search just does not work.
|
| Apple went with deep AI integration into OS in hope to sell more
| devices and Microsoft went full blown corporate + windows
| devices. Google can try to integrate it maybe with Chrome OS to
| sell more devices? They are also trying with various providers
| (like Samsung) but nature of Android is that people might just go
| with basic apps and stuff.
| nolist_policy wrote:
| Don't forget Google Docs, Drive, Photos, GMail and Chrome.
| chucke1992 wrote:
| The problem is that none of Google's products are unified
| into a fully used ecosystem.
|
| For example with Apple they integrated everything together
| seamless into the OS - granted I am not sure if people are
| using their email app or calendar that often. But with AI
| they integrated it all together at least.
|
| With Microsoft they benefit from their tight integration
| between Office suite, Outlook, teams (and calendar
| integration between outlook and teams is quite convenient)
| etc. They only have issues with consumer products as they are
| unable to achieve the same level of integrations as they
| achieve within the corporate - corporate Windows instances
| with laptops and stuff are corporate to Apple products for
| consumers. Microsoft does not have user facing products, but
| their enterprise solutions are nicely connected to each. And
| new services like Loop or Copilot are just naturally expanded
| on that.
|
| But Google? I literally use Gmail but only for emails. Chrome
| for browsing but I have no integrations between Chrome and
| Gmail aside the account overall in my flow. They have their
| streaming service with Youtube Premium but it is not really
| that connected to overall other infra or services - unlike
| for example Apple, that is offering their Apple One
| subscription. App Stores? Google Play exists in its own
| universe that has no relation to other google services
| either. And that's without AI stuff.
|
| There is something missing between google services.
| hot_gril wrote:
| I didn't think the AI hype was overdone until I saw AAPL jump
| so much after announcing AI features. Looks nice, but it's not
| going to make anyone switch to iPhone, and there's no lock-in.
| iMessage must be way more valuable, and that advantage is maybe
| going away.
| chucke1992 wrote:
| That's the thing - apple stocks raised due to AI features but
| it is basically a speculation - if it does not move sales of
| iPhones, the stock will go down.
|
| I think MSFT is the best positioned long term - especially
| when they start producing their own chips - as they have the
| right moat to vendor lock in. Add to that the fact that AWS
| missed the AI boat completely and their could gain market
| share from AWS too.
| amunozo wrote:
| IMO, older DeepMind research was much more exciting and original
| than nowadays. We do not need more LLMs, we need creativity.
| a_bonobo wrote:
| Looking back, I feel like the AlphaFold 3 launch a month ago was
| a precursor to this move. The public-facing side of AlphaFold 3
| ('AlphaFold Server') is severely constrained; if you want the
| novel parts around drug binding prediction you need to pay
| Isomorphic Labs instead.
|
| https://blog.google/technology/ai/google-deepmind-isomorphic...
|
| I expect other developments to follow suite: a bit of R&D with a
| lot of hype and commercialisation.
| vladsanchez wrote:
| http://archive.today/zsnps
| iamleppert wrote:
| Good on them for cracking the whip on those intellectuals and
| extracting value out of them. Even smart people don't get a pass
| when it comes to driving shareholder value and fast ROI.
| theGnuMe wrote:
| This is most likely a tax thing.
___________________________________________________________________
(page generated 2024-06-18 23:02 UTC)