[HN Gopher] Big tech wants to make AI cost nothing
___________________________________________________________________
Big tech wants to make AI cost nothing
Author : LarsDu88
Score : 74 points
Date : 2024-07-24 17:15 UTC (5 hours ago)
(HTM) web link (dublog.net)
(TXT) w3m dump (dublog.net)
| LarsDu88 wrote:
| My musings on the recent commoditization of large scale LLMs
| pjkundert wrote:
| The best way to "bake in" a set of biases into widely available
| AIs, is to make it prohibitively expensive for alternatives
| (without those biases) to be trained.
|
| Unbiased AI is, I believe, an existential threat to the "powers
| that be" retaining control of the narrative, and must be avoided
| at all costs.
| contagiousflow wrote:
| How would you define unbiased?
| crackercrews wrote:
| Good question. For a start, don't pretend that Nazi soldiers
| were a multiracial bunch. And don't do whatever Google did to
| generate clearly-incorrect output like this.
| cthalupa wrote:
| Sure. That's a major over-correction. But there are
| existing and known biases in data sources that need to be
| accounted for - you don't want to further perpetuate those.
|
| I think it's obvious that Google went to a ridiculous
| extreme in the other direction, but there does need to be
| some amount of work done here. For example, we repeatedly
| have seen that just changing the name on a resume to
| something more European sounding can have significant
| impact on callback rates when applying to a job, and if you
| trained a model to screen resumes based on your own resume
| result data, this bias could be picked up by the model.
| That's the sort of situation these are meant to correct
| for.
| dtjb wrote:
| I don't see how you can solve that problem without
| inserting bias in a different direction.
| simonw wrote:
| That multiracial Nazi soldiers thing wasn't baked into the
| model: it was a prompt engineering mistake, part of the
| instructions that a product team were feeding into the
| Gemini consumer product to tell it how to interact with the
| image generation tool.
|
| Here's a similar example from the DALL-E system prompt:
| https://simonwillison.net/2023/Oct/26/add-a-
| walrus/#diversif...
| pjkundert wrote:
| "mistake"
|
| You keep using that word. I don't think it means what you
| think it means.
|
| But seriously; a "mistake" is usually something that
| cannot be foreseen by a group of people reasonably
| talented in the state of the art.
|
| This product release was so far from a "mistake", that it
| isn't funny. It was spectacularly well tested, found to
| be operating within design parameters, and was released
| to great fanfare.
|
| They expressed delight in their product, and actually
| seemed _surprised_ that there was a backlash by the great
| benighted unwashed masses of their lessers, who clearly
| couldn 't be expected to understand the elevated insights
| being produced by their creation!
|
| So: not a "mistake". Institutional Bias, baked into a
| model. Remember: a system's purpose is what is _does_ ,
| not what you think it is supposed to do.
| ryandrake wrote:
| Can you be more specific? Who are the "powers?" What is "the
| narrative?" and why do these "powers" want "control of the
| narrative?"
|
| This just seems like a vague X-Files conspiratorial statement
| without those details.
| dredmorbius wrote:
| Even granting that, the overall point about subsidised / low-
| cost-leader informational content being potentially
| problematic is a fair one to make.
|
| It's one of the chief problems of competing on price
| _generally_ , and particularly so in the case of
| informational exchange.
|
| I'm relatively confident I'd disagree on at least some of
| OP's classifications of biased information. I can still agree
| with their general point all the same.
|
| And in either case, coming up with ways of testing for bias,
| and eliminating _counterfactual_ biases, in AI outputs and
| systems, would _I sincerely hope_ be a Good Thing.
|
| (Though in writing that I suddenly have my own set of doubts,
| we've been fooled before....)
| dexwiz wrote:
| Do you mean unbiased or not biased towards forces in power?
| Everything will have some bias to it, will it not? Even if the
| model does not, surely the training material.
| AlexandrB wrote:
| > Unbiased AI is, I believe, an existential threat to the
| "powers that be" retaining control of the narrative, and must
| be avoided at all costs.
|
| I remember when the internet was supposed to be an existential
| threat to the "powers that be". I'm pretty skeptical of
| narratives like this because the "powers that be" have a lot of
| resources to leverage any new technology for their benefit. At
| _best_ a new technology is gives an asymmetrical advantage to
| small actors for a short time before everyone else catches on.
| mdgrech23 wrote:
| Money talks and if you don't like money they can just throw
| you out of a window or label you a terrorist so you never had
| any real power. Once they flex you've got nothing.
| talldayo wrote:
| "unbiased" in LLM terms means just random token selection. You
| inherently need bias (otherwise called "training data") to
| inform the placement and weight of each token.
|
| Furthermore, unbiased AI isn't likely to be any more usable
| than the garbage we have today. People care about
| hallucinations, model latency, token pricing and other
| practical improvements that can be made. Biases are one of the
| last things stopping people from using AI for legitimate
| purposes; the other issues are far too glaring to ignore.
| impossiblefork wrote:
| Yes, but we understand he what means.
|
| We even have alternative meanings for bias within ML, such as
| for the bias added before non-linearities in many neural
| networks.
|
| He obviously means censored LLMs, and I think his view is
| actually right, although I'm far from sure that these firms
| are in some kind of scheme to produce LLMs biased in this
| sense.
|
| Uncensored, tunable LLMs under the full control of their
| users could scour the internet for propaganda, look for
| connections between people and organisations and just
| generally make the work of propagandists who don't have their
| reader's interests in mind more difficult.
|
| I think we'll end up with that anyway but it's a reasonable
| fear that there'd be people trying to prevent us from getting
| there.
| contagiousflow wrote:
| There is literally no such thing as an unbiased text
| generator. No matter how you cut it there are an infinite
| pool of prompts that will need some sort of implicit value
| system at the heart of the answer. Any implicit bias just
| from selection of training data will be reflected back at
| the user.
|
| > Uncensored, tunable LLMs under the full control of their
| users could scour the internet for propaganda, look for
| connections between people and organisations and just
| generally make the work of propagandists who don't have
| their reader's interests in mind more difficult.
|
| Even this example, what sources do you trust that is or is
| not "in the readers best interest", what is propaganda or
| what is an implicit value in a society, when you tune an
| LLM does that just mean you're steering it to give answers
| that you like more?
|
| Creating an unbiased LLM is as much of a fools errand as
| creating an unbiased news publication
| impossiblefork wrote:
| There is the model that you end up fine-tuning, which
| produces reasonable continuations of almost anything in
| its training dataset, whether it is something any
| approves of or not.
|
| >Even this example, what sources do you trust that is or
| is not "in the readers best interest", what is propaganda
| or what is an implicit value in a society, when you tune
| an LLM does that just mean you're steering it to give
| answers that you like more?
|
| You tune the model yourself. You tune it to find the
| things you're looking for and which interest you.
|
| >Creating an unbiased LLM is as much of a fools errand as
| creating an unbiased news publication
|
| It's what you do before pretraining. You model human-
| written texts with metadata and context with the intent
| of actually modeling those texts, rather than excising
| something which isn't just causing the model to fail to
| learn other things.
|
| It's like, asking "what's a cake, really". We can argue
| about lines etc., but everbody knows. An unbiased
| language model is a reasonable thing to want and it's not
| complicated to understand what it is.
|
| Can you imagine unbiased courts, as an ideal? Somebody
| who just doesn't care about anything other than certain
| things? Just as such a thing can be imagined, so can you
| imagine someone who doesn't about reality and just wants
| to understand human texts.
| dredmorbius wrote:
| I think it's fair to interpret OP's point as an _intentional_
| bias. That might be guard rails, it might be an ideological
| or political viewpoint, it might be something else.
|
| But the notion of promoting a viewpoint _and distributing it
| freely_ is as old as myths and sagas, it 's at the heart of
| propaganda (and is why propagandistic "news" sources are
| often cheap or free to access, often heavily subsidised
| elsewhere).
|
| This isn't to say that _all_ subsidised and low-cost
| information is propaganda, or that all paid-for information
| isn 't. But you should probably squint hard when accessing
| the freely-available stuff, and perhaps make use of several
| largely-independent sources in making assessments.
| pixelready wrote:
| While I take your point, I don't think we're quite at the
| enshittification phase of LLM products yet, where some
| combination of crass marketing and Orwellian narrative control
| are baked into to the models for profit and/or to curry favor
| with govt. Right now we are at a phase where the platform
| companies are still concerned about finding and selling basic
| use cases before the hype bubble bursts. There is still a
| strong possibility that LLM as a product is essentially
| stillborn in the market writ large, because we are trying to
| use these tools where end users expect a perfectly accurate,
| repeatable, deterministic response and there is no guarantee
| that any of these current techniques, cool as they are, will
| cross the threshold of user expectations and utility enough to
| justify the costs. Neither AI doomers nor boosters are accurate
| representations of the general public.
| bearjaws wrote:
| It's already a race to the bottom.
|
| Deepseek v2 lite is a damn good model that runs on old hardware
| already (but its slow).
|
| In 2-3 years we will likely have hardware that runs 70b parameter
| models with enough speed that you will run it locally.
|
| Only when you have difficult questions will you actually pay.
|
| For example I already use https://cluttr.ai to index my screen
| shots and it costs me $0.
|
| (I made this tool tho)
| metadat wrote:
| Neat idea. Is there a self-hostable version of this? I can't
| upload my screenshots to a 3rd party, because they contain lots
| of proprietary info.
| quaintdev wrote:
| Somebody made it for searching memes few days ago
|
| https://www.reddit.com/r/selfhosted/comments/t33rx5/just_rel.
| ..
| bearjaws wrote:
| Totally understand, and it can't be local first without OSS.
| I'll be making it OSS with an Electron App soon.
| countvonbalzac wrote:
| You misspelled screenshots in the title of your website.
| bearjaws wrote:
| Derp lol, I'll fix it when I get home
| ugh123 wrote:
| I think thats a great use case put AI tasks in the background
| for document processing and indexing into structured data.
| osigurdson wrote:
| The title should include a question mark, as they don't appear to
| know either.
| onemoresoop wrote:
| "Why" was stripped from the title
| LarsDu88 wrote:
| Author here. I had a brain farther when posting this to
| Hackernews.
|
| It should have Why in the posting title
| d1sxeyes wrote:
| Seems like you had another one when writing "fart"
| LarsDu88 wrote:
| Damn this autocomploot
| Rumudiez wrote:
| HN's title sanitizer might have automatically stripped it
| einpoklum wrote:
| Use of LLMs/AI costs energy, and the public is indirectly paying
| for it one way or another.
|
| Generalizing somewhat but focusing on a single company:
|
| https://www.statista.com/statistics/788540/energy-consumptio...
|
| In 2022, Google consumed 22.3G Watt-Hours of energy.
|
| Total electricity consumption by humanity:
|
| https://www.statista.com/statistics/280704/world-power-consu...
|
| In 2022, it was 26T Watt-Hours.
|
| Now, Google is a single company, and if we extrapolate with some
| cocktail-napkin math, let's say that similar tech giants put
| together consume, say, 20x Google? 50x Google? So between 2% and
| 6% of all human electricity consumption.
|
| I realize that's not broken down for AI, but I'm sure if we do
| break it down we'll find that's an increasing fraction. In this
| article:
|
| https://cse.engin.umich.edu/stories/power-hungry-ai-research...
|
| the quoted figure is 2% of US electricity usage.
| marinmania wrote:
| For what it's worth energy and electricity consumption are not
| the same. Fuel (for autos, planes, heating oil, etc.) would be
| included in energy consumption but not electrical.
|
| I'm having trouble finding reliable data quickly, but looks
| like 35% of energy consumption in the world is electrical.
|
| It's still an increasing fraction as you say, but it seems like
| a doubling or quadrupling of energy used for AI would probably
| have much less of an impact than the share of the population
| using ice-cars changing a few percentage points.
| einpoklum wrote:
| That's a good point, I'm not sure whether the energy use
| stats I linked really cover non-electricity fuel use. Maybe
| they do, but then there's the question of how much of
| _Google_'s energy use is non-electric, as opposed to
| everybody's consumption.
| coliveira wrote:
| The big problem with modern tech is the increasing separation
| between energy use and its results. In the past, one would buy
| several gallons of gasoline to burn in a car. Nowadays a
| similar amount of the energy is spent far away and only the
| results (Google search of ChatGPT results) are seen.
| edmundsauto wrote:
| It's not so much about using energy - everything uses energy!
| Crypto, for example, uses around 70TWh. Assuming Meta + Google
| + Amazon use similar amounts of energy, there's a lot more
| value produced than in crypto mining.
|
| My point is its impossible to evaluate energy usage without
| considering benefits. For example, heating is one of the
| world's biggest consumers of energy - what % is due to people
| not wanting to wear a sweater inside?
| cut3 wrote:
| Because its something very useful that everyone can benefit from
| like a mobile phone. Its a great value add. If you capture users
| now, you can extract revenue later at scale. Early bird.
| rbirkby wrote:
| Google is to Android as Meta is to Llama. But this time it's all
| about AR glasses.
| franczesko wrote:
| Open source alternatives means that it will become difficult or
| close to impossible to monetize llms. It also means that no clear
| hegemony will occur. Fragmentation might be very beneficial for
| the status quo.
| badgersnake wrote:
| There is no open source LLM. What even is that? LLMs don't have
| source, they have training data and they sure as hell aren't
| giving that away for free. Even if they did you'd need to
| remortgage to rent the GPUs.
| agentultra wrote:
| How is this even remotely sustainable let alone so low-margin
| that it could be "given away for free"?
|
| The only reason folks are _only_ paying a small monthly
| subscription for gippity is literally because of all the VC money
| flowing in. Training, running, and scaling this stuff has a huge
| cost. The extra datacentres, the fresh water, all the air
| conditioning, energy usage, chips, the exploited unprotected
| labour, etc. It seems very expensive.
|
| Usually when people selling shovels are giving shovels away for
| free they're banking on a payoff. Usually a regulatory capture
| payoff. Or a hedge of some kind?
|
| I'm still trying to wrap my head around this.
|
| _Update_ : .. VC money flowing in and the extremely favourable
| taxation "exceptions" for tech companies in thirsty economies...
| mschuster91 wrote:
| > Usually when people selling shovels are giving shovels away
| for free they're banking on a payoff. Usually a regulatory
| capture payoff. Or a hedge of some kind?
|
| The payoff is different... everyone's banking on _someone_
| making a breakthrough regarding AGI /ASI and then being the
| first one to actually make a mass market product out of it that
| achieves dominance.
|
| But for that to happen, you need _a lot_ of extremely smart
| people working for you, and you get these people working for
| you by giving them something to play around with.
| meiraleal wrote:
| That's the best interpretation of Meta move I've read so far.
|
| I guess it is Meta's Chrome moment.
| bookaway wrote:
| The article's title is terrible. The article itself seems to have
| all the right pieces but connects them in weird ways.
|
| Big tech wants to make AI cost nothing to _end-users_ maybe, but
| Google and Microsoft want the cost of _hosting_ AI to make your
| eyes bleed so you don 't compete or trim any profits off their
| cloud services. As the article points out Facebook does not offer
| cloud services so its interests in this case align with mom and
| pop shops that don't want to be dependent on big tech for AI.
|
| But Mistral was way more useful to mom and pop shops when they
| were trying to eke out performance from self-hostable small
| models. Microsoft took them out of that game. These enormous
| models may help out boutique data center companies to compete
| with big tech's cloud offerings but it's beyond a small dev shop
| who wants to run a co-pilot on a couple of servers.
|
| Microsoft and Google don't want you to learn that a 7B model can
| come close to a model 50x-100x its size. We don't know that's
| even possible, you say? That's right we don't know, but _they don
| 't even want you to try and find out if it's possible or not_.
| Such is the threat to their cloud offerings.
|
| If they did Microsoft would have made a much bigger deal of
| things like their Orca-math model and would have left Mistral
| well alone.
| stavros wrote:
| How did Microsoft take Mistral out? I missed that.
| bookaway wrote:
| Not "take them out", I said they took them out of the small
| model game. They did that by giving Mistral free compute, so
| Mistral turned its main focus to large models. Their
| announcement after the Microsoft deal was literally "Mistral
| Large".
|
| [0] https://news.ycombinator.com/item?id=39511530
|
| [1] https://news.ycombinator.com/item?id=39512683
| stavros wrote:
| I see, thanks. I didn't know they got free compute.
| imjonse wrote:
| MS and Google keep releasing small models like Phi and Gemma, I
| don't think they want to hide the fact these can be very
| capable.
| bookaway wrote:
| The Gemma of Clement Farabet the French-American student of
| Meta's French-American LeCunn? I wouldn't be surprised if
| they share an ideological affinity. The big players allow
| some breathing space for the top brass so they don't walk.
| Look at the timing of Andrej Karpathy departure from OpenAI.
| I'm sure he might give a lot of different public answers, but
| it just so happened that it coincided when the oxygen started
| getting a little stale around there. [0]
|
| This is the forest for the trees situation. The correct
| analogy is Chrome and ad blockers. Google didn't tighten the
| screws until the bean counters started saying it was starting
| to bite.
|
| [0] https://xcancel.com/karpathy/status/1812983013481062761
| LarsDu88 wrote:
| Author here:
|
| The title "Why big companies like Meta want to commoditize open
| source and open weight models to increase demand for their
| complimentary services" did not quite have the same ring to it
| to be quite honest
| bookaway wrote:
| Love the clarification, thanks!
| meiraleal wrote:
| > in this case align with mom and pop shops
|
| But only until. Facebook rugpulled even React, a (once in the
| past) javascript library. I can't wait to see what they will
| pull out when they become the AI overlord.
| 2OEH8eoCRo0 wrote:
| Nothing in the world's more expensive than free.
| riffic wrote:
| when a service is free, who is the customer and who is the
| product being sold?
| myhf wrote:
| The current hype cycle is about artificially inflating the
| value of Nvidia stock by creating do-nothing "products" and
| "customers". It would be more efficient to have the public pay
| for GPUs directly.
| LarsDu88 wrote:
| Author here:
|
| For cloud hosts like Amazon, Google, and Microsoft, the product
| is their inference hardware (mostly GPUs). The bigger the open
| model, the better!
|
| For Meta however, its a bit more mysterious. From what I can
| glean, the stated objective is to ensure that the world
| standardizes on the same AI platform tooling Meta uses
| internally (presumably to drive down dev costs?).
|
| The bigger "product" however is content creation itself. Giving
| users the ability to generate engaging content more easily can
| keep folks using social media and buying ads.
| viccis wrote:
| >nation-states like China
|
| Is there a reason these articles like to say "nation-state"
| rather than "countries"? I think the question of whether China is
| a nation-state is not entirely settled (the state is broader than
| the nation in its case), but also it seems odd to exclude
| countries like Belgium that are not nation-states from these
| kinds of statements.
| dijit wrote:
| I agree, it's poor vernacular.
|
| What they mean, of course, is an extremely well funded-
| government sponsored, tech agency acting as an aggressive
| security service.
|
| IE; NSA, GCHQ.
|
| As opposed to a tech agency that is acting inwards to the
| country.
|
| IE; CIA, MI5 (Security Service).
|
| And opposed to a self-funded hacktivist group.
|
| I would greatly prefer better nomenclature though. Nation-State
| is almost a non-term, since every nation is a state,
| effectively. - Normally I have seen "state-sponsored", which
| denotes the correct meaning.
|
| In computer science we tend to have quite clear names for
| things (blue team, red team). Maybe someone could come up with
| something better?
| viccis wrote:
| >Nation-State is almost a non-term, since every nation is a
| state, effectively.
|
| This is not true, especially outside of Europe and the US.
| Even within Europe, Belgium is not a nation-state for
| example. Many African countries are not either, with Nigeria
| being an easy example.
|
| State-sponsored is a much better term, I agree, as the
| phrases are typically used to describe state activity.
| sky-wa1ker wrote:
| I don't think the author cares about semantics here, "nation-
| states" is in direct contrast with "larger companies" used in
| previous line. Using a single word "countries" may not sound
| sufficient to contrast the pair "larger companies" and may not
| give it emphasis needed to indicate the gravity of the
| comparison being made. It had to be this or something like
| "entire countries".
___________________________________________________________________
(page generated 2024-07-24 23:10 UTC)