[HN Gopher] I Spent a Week with Gemini Pro 1.5-It's Fantastic
___________________________________________________________________
I Spent a Week with Gemini Pro 1.5-It's Fantastic
Author : dshipper
Score : 96 points
Date : 2024-02-23 15:23 UTC (7 hours ago)
(HTM) web link (every.to)
(TXT) w3m dump (every.to)
| platelminto wrote:
| GPT-4 Turbo has a context window of 128k tokens, not 32k as the
| article says.
| dshipper wrote:
| Fixed!! Thanks for finding that
| Workaccount2 wrote:
| I believe with the API you can get 128k, but using the app or
| web client it is 32k. This might have changed though.
| criddell wrote:
| I kind of love the idea of feeding the text of entire books to an
| AI. More often than I'd like to admit, I'll be reading a novel
| and find myself not remembering who some character is. I'd love
| to be able to highlight a name in my ereader and it would see
| that I'm 85 pages into Neuromancer and give me an answer based on
| that (ie no spoilers).
|
| Or have a textbook that I can get some help and hints while
| working through problems and get stuck like you might get with a
| good study partner.
| TrueGeek wrote:
| I'm the opposite. In movies (and real life to an extent) I have
| trouble telling people apart. I'd love a closed caption like
| service that just put name tags over everyone.
| ryandrake wrote:
| That's one of the perpetually-SciFi use cases of Augmented
| Reality: You're walking around with AR glasses and whenever
| someone comes into your field of view, your display hovers
| over them their name, their job, when you last talked to
| them, what their hobbies are, and so on. A huge privacy
| nightmare, but useful for people who can't recall names and
| details about people they've met.
| egeozcan wrote:
| I don't think it'd be a privacy nightmare if it used
| private databases created by the user. Like a personal CRM,
| something like Monica, but with a real-time touch.
| avarun wrote:
| Wouldn't be from a legal sense, but the societal
| implications of technology like that becoming commonplace
| are still immense. The limitations of human memory
| provide safety in a way today that would quickly erode if
| everybody could remember exactly everything that's ever
| been said or seen around them.
| chambored wrote:
| Safety from what exactly?
| avarun wrote:
| "You said X 3 years ago, but now you said, which is the
| opposite of X. How dare you?" is one class of problems.
| Another is that you can learn quite a bit more about a
| person than they wished to actually divulge to you if
| you're able to capture and study their exact behaviors
| over a long enough stretch of time.
| twerkmonsta wrote:
| Wait, why are people not allowed to change their mind on
| something? If anything this would make it more explicit
| and understandable when people did change their mind on
| something.
| groggo wrote:
| Still privacy nightmare and creepy. There's plenty of
| public info on people, that once collected and assembled
| into one place is basically stalking. Not saying it's not
| a cool idea though :)
| pmarreck wrote:
| I had a product idea for an AR app that would do this for
| everyone who's opted into it. So for real-world networking
| events, you might choose to disclose some things about
| yourself but only for that venue and only for some window
| of time for example.
|
| I never built it, but it's perfectly possible to do.
|
| The genius idea IMHO was the business model- If you were
| into certain things you wanted to keep private from most
| but only wanted to disclose to other people who were into
| those same things, you could pay a fee, and it would then
| show you others who were in that "market" (of ideas, drugs,
| sex, whatever). (It might only ask you to pay it if it
| found someone nearby who matched. And then it would
| automatically notify the other person unless you paid an
| ADDITIONAL fee... Not sure about the latter idea, but it
| was an idea.)
|
| The only issue is everyone holding their phone up in front
| of their faces.
| HKH2 wrote:
| How would you stop spies or undercover cops trying to
| infiltrate the "market"?
| pmarreck wrote:
| Or people who want to "out" gay people. I know.
|
| That would be a good argument over not permitting a
| unilateral notification of a match (which, at the very
| least, I wanted to make very expensive and thus
| profitable, if it's allowed at all). If it notified both
| people 100% of the time, and one of you was a possible
| impostor, you could report them. And from a legal
| standpoint, showing interest in a market doesn't make you
| guilty. And, you could possibly also build "cred" in one
| of these tagged "markets" by getting cred from others who
| say you're legit, and that information would be revealed
| at the same time (possibly at your discretion).
| piva00 wrote:
| > The genius idea IMHO was the business model- If you
| were into certain things you wanted to keep private from
| most but only wanted to disclose to other people who were
| into those same things, you could pay a fee
|
| > The only issue is everyone holding their phone up in
| front of their faces.
|
| No, the genius idea is its major issue, just by paying
| you gain access to private data (people's preferences)
| without any kind of chain of trust to make sure that
| someone is actually part of the group ("market" in your
| terms) for which they want access to.
|
| By paying you could know that someone around you is
| looking for cocaine, or is willing to sell sexual
| services, or is looking to match other people from the
| same gender, or holds a certain political view against an
| authoritarian government, etc.
| pmarreck wrote:
| I answered this in a sibling comment. You could acquire
| credibility in a particular preference from the network
| over time.
|
| https://news.ycombinator.com/item?id=39482786
| HKH2 wrote:
| And instead of just shrugging it off, you could tag
| strangers that annoy you and end up with a giant list of
| grudges against a whole host of people. The false positives
| (e.g. twins and doppelgangers) should make it interesting.
| EwanG wrote:
| Which feeds into Saint Motel's song "You're Nobody Til
| Somebody Wants You Dead" which has a bit about how the
| list just grows and grows until it's everyone you've ever
| known...
| dylan604 wrote:
| Take it to the next step towards Black Mirror where the
| AR shadows out people you've blocked and then mutes their
| voice so you can't hear them
| earthnail wrote:
| That would make for fantastic comedic situations when you
| then physically bump into them after you erased them from
| your AR vision xD.
| BurningFrog wrote:
| I've seen scenes in movies where assistants of heads of state
| will discreetly whisper to them who the people in the room
| are.
|
| With a service like this we could all live like Kings!
| avarun wrote:
| This features distinctively in the show Veep, where one of
| the main characters provides exactly this for the Vice
| President.
| eesmith wrote:
| https://en.wikipedia.org/wiki/Farley_file
|
| > A Farley file is a set of records kept by politicians on
| people whom they have met.
|
| > The term is named for James Farley, Franklin Roosevelt's
| campaign manager. Farley, who went on to become Postmaster
| General and chairman of the Democratic National Committee,
| kept a file on everyone he or Roosevelt met.
|
| > Whenever people were scheduled to meet again with
| Roosevelt, Farley would review their files. That allowed
| Roosevelt to meet them again while knowing their spouse,
| their children's names and ages, and anything else that had
| come out of earlier meetings or any other intelligence that
| Farley had added to the file. The effect was powerful and
| intimate.
|
| > Farley files are now commonly kept by other politicians
| and businesspeople.
| gniv wrote:
| Amazon already does some of this (they identify actors in a
| scene iirc), so they could "easily" extend it to what you're
| suggesting.
| esafak wrote:
| Xray, they call it. It's a great feature!
| https://www.amazon.com/salp/xray
| mncharity wrote:
| Many years ago there was an MIT startup based on the idea,
| IIRC, that subliminally flashed names increased recall among
| cognitively impaired elderly when the flashed names were
| correct, but didn't negatively impact recall when the flashed
| names were incorrect. So even quite poor face recognition
| could be worthwhile.
| crazygringo wrote:
| That's a truly fantastic idea actually. I'd _love_ to see that
| built into e-readers.
|
| As well as when I pick back up reading after two weeks --
| remind me everything that's happened so far? Where it gives a
| five-paragraph summary where the first paragraph is top-level
| covering the entire story so far, and the last paragraph is
| just about the previous few pages.
|
| Not to mention with non-fiction -- highlight an acronym that
| was defined somewhere 28 pages ago and tell me what the heck it
| is again?!
| glhaynes wrote:
| I love these ideas. One more: "He said xyz but we know he's
| lying. What would motivate him to do that?"
| mbil wrote:
| Kindle X-Ray kind of does this
| https://goodereader.com/blog/kindle/what-is-x-ray-for-the-am...
| world2vec wrote:
| Great idea, especially with huge books with hundreds of
| characters (looking at you "Gravity's Rainbow" and your ~400
| characters with wacky names).
| wouldbecouldbe wrote:
| For most classic novels I expect GPT to already have in it's
| memory.
| cryptonector wrote:
| Soon we'll have AI writing books, then reading them for us so
| we don't have to.
|
| There is value to that, if we mostly only use this capability
| to digest books we otherwise wouldn't read but also if we don't
| stop reading books. Most likely we'll just stop reading books,
| and that strikes me as scary.
| keiferski wrote:
| FYI someone did do this for Neuromancer. Not sure if they used
| AI or not.
|
| https://docs.google.com/document/u/0/d/1ovTscY-bEuMNAEgNXTCX...
| jamie_ca wrote:
| I can't recall which reader app I used, but I've seen this done
| before in ages past.
|
| No AI, so no big concept of inferred identities, but if
| someone's referenced by name you could long-tap the name, and
| get a list of all previous references in the text. Super useful
| for referencing secondary characters in super-long epics like
| The Wheel of Time.
| pests wrote:
| I'm really bad with names. I almost wish for unique color
| highlighting for every name. I would remember previous scenes
| or conversations way better than keeping track of tons of
| character names.
| ChildOfChaos wrote:
| I'd love to feed it all the advice books on certain topics that
| I am struggling with and then chat with it like a group of
| advisors.
| Sakos wrote:
| I love the potential of having such a big context window, but I'm
| concerned about who will get access to it (or rather who won't
| get access to it) and what it will cost or who will pay for it.
| criddell wrote:
| You could have asked the same question in 1964 when IBM
| released the System/360. Nice computer, but who will pay for it
| and who will have access to it?
|
| I think it's inevitable that these AI's will end up costing
| almost nothing to use and will be available to everybody. GPT-4
| is already less than $1 / day and that's only going to go down.
| Sakos wrote:
| I didn't ask it then. I wasn't even alive. I'm asking now. It
| is a legitimate concern and continues to be a concern with
| these AI models and you whining that "oh but 50 years ago bla
| bla" (as if these two things are in any way comparable)
| doesn't change anything about the risks of selective access.
| eesmith wrote:
| I do not think that's a good example. A lot of people jumped
| on the chance to buy a 360. Here are some quotes from https:/
| /archive.org/details/datamation0064unse_no5/page/68/m... :
|
| > The internal IBM reaction could be characterized as quiet,
| smug elation. One office is supposed to have sold its yearly
| quota on A (Announcement) -Day. In Texas, a man allegedly
| interrupted the 360 presentation to demand he be allowed to
| order one right then . . . which sounds like a combination
| plant and a new version of the rich Texan jokes. ...
|
| > the 360 announcement has to worry the competition
| considerably . . . partly because anything new from IBM
| creates an automatic bandwagon effect, partly because the
| completeness of the new line offers less reason for people to
| look outside. ...
|
| > another feels that the economic incentive (rental cuts of
| 50 per cent for 7080, 7090) will force him down the 360
| route. And he thinks 360 interrupt features will open the
| door to real-time applications which can be approached on an
| incremental basis impossible before. ...
|
| > One maverick doesn't share the enthusiasm of his company,
| which ordered "plenty" of 360's within an hour of the
| announcement, without price agreements.
|
| And from https://archive.org/details/bitsavers_Electronic9640
| 504_9809... :
|
| > Other computer manufacturers profess to be unworried, but
| International Business Machines Corp. has received millions
| of dollars in orders for its system/360 computer
| [Electronics, April 20, p. 101].
|
| Here are some of the people who paid for it in 1964:
|
| https://archive.org/details/sim_computers-and-
| people_1964_13...
|
| > "Men's Fashion Firm Orders IBM System/360 Computer," 13/7
| (July)
|
| > "Score of IBM System/360's to be Leased by Paper Company,"
| 13/8 (Aug.)
|
| The Albert Government Telephone Commission bought two: https:
| //archive.org/details/annualreportofa1964albe_0/page/1...
| neolefty wrote:
| How does it scale to such a large context window -- is it
| publicly known, or is there some high-quality speculation out
| there that you recommend?
| mbil wrote:
| I don't know, but there was a paper posted here yesterday:
| LongRoPE: Extending LLM Context Window Beyond 2M Tokens[0]
|
| [0]: https://news.ycombinator.com/item?id=39465357
| alphabetting wrote:
| Not publicly known. I think speculation is use of mamba
| technique but I haven't been following closely
| cal85 wrote:
| Not sure about high quality, but they discussed this question a
| bit on the recent All-In Podcast.
| rfoo wrote:
| My bet is it's just brute force.
|
| I don't understand how they did 10M though, this isn't in the
| brute-force-with-nice-optimizations-on-systems-side-may-do-it
| ballpark, but they aren't going to release this to the public
| anyways so who knows, maybe they don't and it actually takes a
| day to finish a 10M prompt.
| coldtea wrote:
| > _That will be forgotten in a week; this will be relevant for
| months and years to come._
|
| Or, you know, until next month or so, when OpenAI bumps their
| offer
| 4bpp wrote:
| I imagine the folks over at NSA must be rubbing their hands over
| the possibilities this will open up for querying the data they
| have been diligently storing over the years.
| croes wrote:
| That's the point where hallucinations are pretty dangerous.
| pmarreck wrote:
| Not too hard to verify.
| dylan604 wrote:
| Verify? There are plenty of examples where things have been
| "verified" to prove a point. WMDs ring a bell?
| pmarreck wrote:
| What is your point? That obtaining absolute knowledge of
| truth is impossible, and therefore anything claiming to
| be true is worthless?
|
| In general, be careful not to kill "good" on the way to
| attempting to obtain "perfect" in vain. And GPT4's
| hallucination rate is quite low at this point (may of
| course depend on the topic).
| dylan604 wrote:
| Not at all. I'm coming from the opposite side saying that
| anything can be "verified" true if you just continue to
| repeat it as truth so that people accept it. Say often,
| say it loud. Facts be damned
| pmarreck wrote:
| Yes but that's the argument by repetition (ad nauseam)
| fallacy
| avarun wrote:
| Palantir already provides this product to them.
| seanmcdirmid wrote:
| They could always try using tech to reduce their false positive
| rate rather than increase it.
| jeffbee wrote:
| NSA does not have large-scale storage. If they did, it would be
| in a building somewhere, it would have electric power and they
| would have bought storage devices for it from some company. The
| largest-known actual NSA datacenter is the size of a runty
| leftover from having built a real cloud datacenter, and there's
| only one of those while FAMG have hundreds of far larger
| datacenters.
| CobrastanJorji wrote:
| Phew, glad the NSA doesn't have any large scale storage. Big
| relief. By the way, what do they use their $10 billion AWS
| contract for?
| jeffbee wrote:
| I know it's not storing every email, phone call, photo, and
| video ever transmitted on the internet, like certain people
| want you to believe.
|
| A $10b AWS contract would not even amount to enough storage
| to keep one copy of the public web.
| wkat4242 wrote:
| Wouldn't that cost a fortune? If I feed the maximum into gpt-4 it
| will already cost $1.28 per interaction! Or is Gemini that much
| cheaper too?
| alphabetting wrote:
| I think Google has some big advantages in cost with TPUs and
| their crazy datacenter infra (stuff like optical circuit
| switches) but I'd guess long context is still going to be
| expensive initially.
| wkat4242 wrote:
| Yeah I'm specifically interested in this because I'm in a lot
| of local telegram groups which I have no patience to catch up
| on every day. I'd love to have ChatGPT summarise it for me
| based on a list of topics I care about.
|
| Sadly the cost of GPT-4 (even turbo) tends to balloon for
| this usecase. And GPT-3.5-turbo while much cheaper and more
| than accurate enough, has a context window that's too
| shallow.
|
| I wonder if Telegram will add this kind of feature also for
| premium users (which I also subscribe to) but I imagine it
| won't work at the current pricing levels. But it would be
| nice not having to build it myself.
| croes wrote:
| I'm a bit worried about the resource consumption of all these
| AIs. Could it be that the mass of AIs that are now being created
| are driving climate change and in return we are mainly getting
| more text summaries and cat pictures?
| pmarreck wrote:
| Instead of complaining to a void about resource consumption,
| you should be pushing for green power, then. Resource
| consumption isn't a thing that is going down, and it most
| certainly won't go down unless there's an economic incentive to
| do so.
| eropple wrote:
| You know that somebody can hold two thoughts in their head at
| once, yeah?
|
| Green power is great! But there'll be limits to how much of
| that there is, too, and asking if pictures of hypothetical
| cats is a good use of that is also reasonable.
| pmarreck wrote:
| It's not. But I'm also making a judgment call and neither
| of us knows or can even evaluate what percent of these
| queries are "a waste."
|
| I'm flying with my family from New York to Florida in a
| month to visit my sister's side of the family. How would I
| objectively evaluate whether that is "worth" the impact on
| my carbon footprint that that flight will have?
| croes wrote:
| Isn't a great part of even green power converted to heat
| while consumed? Isn't that also additional energy which heats
| the atmosphere or is the amount too low for any effects?
| pmarreck wrote:
| Global warming is more about the "greenhouse effect"
| (certain gases like CO2 helping to trap in infrared energy)
| than it is about individual sources of infrared energy.
| svara wrote:
| I'm worried about that too, but it does seem like one of the
| things that can be moved to clean energy the easiest. You can
| route requests to the datacenters where energy is cheap, i.e.
| where generation from renewables is currently high.
| danpalmer wrote:
| Data center infrastructure is a relatively small component of
| global emissions. I believe "compute" is something like <2% of
| global emissions, whereas construction is double digits, travel
| is double digits, etc. AI might increase this, maybe
| substantially, but it's unlikely to 10x it overall, as
| "traditional" compute is going to be dominant for a long time
| to come still.
|
| Add to this the fact that companies in this space tend to be
| significantly better than average on carbon emissions
| commitments. I'm biased as a Googler, but the fact Google is
| entirely carbon neutral is one of the reasons I'm here. This is
| done mostly through buying green energy I believe, so our AI
| stuff is in a pretty good place in this respect, in my opinion.
|
| I think it's reasonable to be a little concerned, but overall I
| don't think AI is going to be a significant contributor to the
| climate crisis, and actually has the potential to help in
| reducing carbon emissions or atmospheric warming in other ways.
| shmageggy wrote:
| It's a valid concern, and there is research into this.
| https://news.climate.columbia.edu/2023/06/09/ais-growing-car...
| is one article, but lots more to be found via Google. Currently
| AI training is very small relative to agriculture and industry,
| but of course it's trending upwards.
| aantix wrote:
| Does the model feel performant because it's not under any serious
| production load?
| EwanG wrote:
| Article seems to suggest just that as the author states that
| he's doubtful the model will perform as well when it's scaled
| to general Google usage
| Solvency wrote:
| How can Google so thoroughly embarrass themselves on the image
| front and then do well on text?
| itchy_spider44 wrote:
| Because engineers make great things and management tells them
| to make it woke
| hersko wrote:
| They are not doing well on text....
|
| https://deepnewz.com/politics/google-s-woke-gemini-ai-slamme...
| wredue wrote:
| Imagine being personally offended by a company realizing
| different skin colours exist, and for an AI trained on not
| historical images to maybe be not quite historical.
|
| It's funny, though, how historical skin colour seems to only
| matter when what you picture as a definitely white person is
| represented as not white, but when a definitely brown person
| (Jesus) is turned white, alls well.
| Solvency wrote:
| ...what?
| rkangel wrote:
| This is exactly the sort of article that I want to read about
| this sort of topic.:
|
| * Written with concrete examples of their points
|
| * Provides balance and caveats
|
| * Declares their own interest (e.g. "LlamaIndex (where I'm an
| investor)")
| hersko wrote:
| > This is not the same as the publicly available version of
| Gemini that made headlines for refusing to create pictures of
| white people. That will be forgotten in a week; this will be
| relevant for months and years to come.
|
| I cannot disagree with this more strongly. The image issue is
| just indicative of the much larger issue where Google's far left
| DEI policies are infusing their products. This is blatantly
| obvious with the ridiculous image issues, but the problem is that
| their search is probably similarly compromised and is much less
| obvious with far more dire consequences.
| mrcartmeneses wrote:
| Mountains != molehills
| dylan604 wrote:
| no, but we're making transformers, right? it's easy to
| transform a molehill into a mountain
| tr3ntg wrote:
| Do you remember Tay? You don't have to have "far left DEI
| policies" to want to defend against worst case scenario as
| strongly as possible. Even if, in the case of this image
| weirdness, it works against you.
|
| Google has so much to lose in terms of public perception if
| they allow their models to do anything offensive. Now, if your
| point was that "the same decisions that caused the image fiasco
| will find its way into Gemini 1.5 upon public release,
| softening its potential impact," then I would agree.
| doug_durham wrote:
| This has nothing to do with the rightwing scary boogieman
| "DEI". This is more about preventing Google's stock from
| getting hammered when White Supremacists use the tool to create
| excellent propaganda. No company can withstand that. The
| current world of context-free social media posting based
| journalism is causing this. Google's approach was heavy handed
| and poorly thought out. I don't know what their solution will
| be.
| zarathustreal wrote:
| "White Supremacists" creating "excellent propaganda"?
|
| "rightwing scary boogieman "DEI""?
|
| My friend, I strongly suggest you take a moment to do some
| self-reflection
| egillie wrote:
| Wait do people not remember this?
| https://twitter.com/rzhang88/status/1549472829304741888?t=R4...
| pickledish wrote:
| > This is about enough to accept Peter Singer's comparatively
| slim 354-page volume Animal Liberation, one of the founding texts
| of the effective altruism movement.
|
| What? I might be confused, is this a joke I don't get, or is
| there some connection between this book and EA that I haven't
| heard of?
| avarun wrote:
| Peter Singer is well known as a "founder" of EA, and
| vegetarianism is a tenet that many EAs hold, whether or not you
| directly consider it part of EA. Animal welfare is, at the very
| least, one of the core causes.
|
| That specific book may have been written before effective
| altruism really existed, but it makes sense for one of Singer's
| books to be considered a founding text.
| pickledish wrote:
| Ahhh ok, had no idea, I'm pretty new to this stuff. Thank you
| for the explanation!
| tr3ntg wrote:
| > These models often perform differently (read: worse) when they
| are released publicly, and we don't know how Gemini will perform
| when it's tasked with operating at Google scale.
|
| I seriously hope Google learns from ChatGPT's ever-degrading
| reputation and finds a way to prioritize keeping the model
| operating at peak performance. Whether it's limiting access,
| raising the price, or both, I really want to have this high
| quality of an experience with the model when it's released
| publicly.
| next_xibalba wrote:
| It is hard to imagine Gemini Pro being useful given the truly
| bizarre biases and neutering introduced by the Google team in the
| free version of Gemini.
| feoren wrote:
| It's _hard to imagine_ that the pro version removes the line
| "oh, and make sure any humans are ethnically diverse" from its
| system prompt?
| next_xibalba wrote:
| I don't understand your question (if it is made in good
| faith). Are you implying that a pro version would allow the
| user to modify the system prompt?
|
| Also, your assumption is that the data used to train the
| model is not similarly biased, i.e. it is merely a system
| prompt that is introducing biases so crazy that Google took
| the feature offline. It seems likely that the corpus has had
| wrongthink expunged prior to training.
| feoren wrote:
| Yes, I'm assuming the forced diversity in its generated
| images is due to a system prompt; no, I don't believe they
| threw out all the pictures of white people before training.
| If they threw away all the pictures of German WWII soldiers
| that were white, then Gemini wouldn't know what German WWII
| soldiers looked like _at all_. No, it 's clearly a poorly
| thought out system prompt. "Generate a picture of some
| German soldiers in 1943 _(but make sure they 're ethnically
| diverse!)_"
|
| They took it offline not because it takes a long time to
| change the prompt, but because it takes a long time to
| verify that their new prompt isn't similarly problematic.
|
| > It seems likely that the corpus has had wrongthink
| expunged prior to training.
|
| It seems likely to you because you erroneously believe that
| "wokeism" is some sort of intentional strategy and not just
| people trying to be decent. And because you haven't thought
| about how much effort it would take to do that and how
| little training data there would be left (in some areas,
| anyway).
|
| > Are you implying that a pro version would allow the user
| to modify the system prompt?
|
| I am saying it is not _hard to imagine_ , as you claimed,
| that the pro version would have a different prompt than the
| free version*. Because I know that wokeism is not some
| corrupt mind virus where we're all conspiring to de-white
| your life; it's just people trying to be decent and
| sometimes over-correcting one way or the other.
|
| * Apparently these are the same version, but it's still not
| a death knell for the entire model that one version of it
| included a poorly thought-out system prompt.
| next_xibalba wrote:
| > you erroneously believe that "wokeism
|
| This is an ironic statement. On the one hand, you are
| able to read my mind and determine the worldview and
| intent behind my words. One the other, you suggest I'm
| doing the same to people who subscribe to "wokeism".
|
| Meanwhile, Jack Krawczyk, a Sr. Director of Product on
| Gemini, has been publicly declaring on X (over years)
| things like "...This is America, where the #1 value our
| populace seeks to uphold is racism" and "...We obviously
| have egregious racism in this country.." and "I've been
| crying in intermittent bursts for the past 24 hours since
| casting my ballot. Filling in that Biden/Harris line felt
| cathartic." Do you think he is an exemplar of "wokeism"
| (however you want to define that term)? Do you think he
| is influential within the Gemini org? Do you think he is
| emblematic of the worldview of Google employees? Do you
| think his words are those of the type of person who is
| "just trying to be decent" but has made honest mistakes
| in his work?
|
| > I am saying it is not hard to imagine,
|
| This is really pretty pedantic, don't you think? I'd bet
| most people who read those words understood what I meant.
| Which is that it is _unlikely_ (though, yes, not hard to
| imagine) that Gemini will allow users to alter the system
| prompt.
|
| The bottom line is, Google appears to have either 1)
| introduced extreme bias into Gemini in some way or 2) to
| be pretty incompetent. Neither inspires confidence.
| losvedir wrote:
| Confusingly, Pro _is_ the free version. Ultra is the paid
| one. What some people have access to here is the next-ish
| generation of Pro, 1.5, which sports a huge context window. I
| haven 't heard anything about an Ultra 1.5 yet.
|
| (As a paying user of Ultra, I'm kind of bummed about not
| having access to this improved Pro...)
| feoren wrote:
| Thanks for the clarification -- that is quite confusing.
| eesmith wrote:
| > I wanted an anecdote to open the essay with, so I asked Gemini
| to find one in my reading highlights. It came up with something
| perfect:
|
| Can someone verify that anecdote is true? Here is what the image
| contains:
|
| > From _The Publisher_ : In the early days of Time magazine, co-
| founder Henry Luce was responsible for both the editorial and
| business sides of the operation. He was a brilliant editor, but
| he had little experience or interest in business. As a result, he
| often found himself overwhelmed with work. One day, his colleague
| Briton Hadden said to him, "Harry, you're trying to do everything
| yourself. You need to delegate more." Luce replied, "But I can do
| it all myself, and I can do it better than anyone else." Hadden
| shook his head and said, "That's not the point. The point is to
| build an organization that can do things without you. You're not
| going to be able to run this magazine forever."
|
| That citation appears to be "The Publisher : Henry Luce and his
| American century".
|
| The book is available at archive.org as searchable text returning
| snippets, at
| https://archive.org/details/publisherhenrylu0000brin_o9p4/
|
| Search is unable to find the word "delegate" in the book. The six
| matches for "forever" are not relevant. The matches for
| "overwhelmed" are not relevant.
|
| A search for Hadden finds no anecdote like the above. The closest
| are on page 104,
| https://archive.org/details/publisherhenrylu0000brin_o9p4/pa... :
|
| """For Harry the last weeks of 1922 were doubly stressful. Not
| only was he working with Hadden to shape the content of the
| magazine, he was also working more or less alone to ensure that
| Time would be able to function as a business. This was an area of
| the enterprise in which Hadden took almost no interest and for
| which he had little talent. Luce, however, proved to be a very
| good businessman, somewhat to his dismay--since, like Brit, his
| original interest in "the paper" had been primarily editorial.
| ("Now the Bratch is really the editor of TIME," he wrote, "and I,
| alas, alas, alas, am business manager. . .. Of course no one but
| Brit and I know this!") He negotiated contracts with paper
| suppliers and printers. He contracted out the advertising. He
| supervised the budget. He set salaries and terms for employees.
| He supervised the setting up of the office. And whenever he
| could, he sat with Brit and marked up copy or discussed plans for
| the next issue."""
|
| That sounds like delegation to me _and_ decent at business _and_
| not doing much work as an editor.
|
| There's also the anecdote on page 141 at
| https://archive.org/details/publisherhenrylu0000brin_o9p4/pa... :
|
| """In the meantime Luce threw himself into the editing of Time.
| He was a more efficient and organized editor than Hadden. He
| created a schedule for writers and editors, held regular
| meetings, had an organized staff critique of each issue every
| week. ("Don't hesitate to flay a fellow-worker's work.
| Occasionally submit an idea," he wrote.) He was also calmer and
| less erratic. Despite the intense loyalty Hadden inspired among
| members of his staff, some editors and writers apparently
| preferred Luce to his explosive partner; others missed the energy
| and inspiration that Hadden had brought to the newsroom. In any
| case the magazine itself--whose staff was so firmly molded by
| Hadden's style and tastes--was not noticeably different under
| Luce's editorship than it had been under Hadden's. And just as
| Hadden, the publisher, moonlighted as an editor, so Luce, now the
| editor, found himself moonlighting as publisher, both because he
| was so invested in the business operations of the company that he
| could not easily give them up, and also because he felt it
| necessary to compensate for Hadden's inattention.""""
|
| Again, it doesn't seem to match the summary from Gemini.
|
| Does someone here have better luck than I on verifying the
| accuracy of the anecdote? Because so far it does not seem valid.
| eesmith wrote:
| To follow up to myself - the author asked:
|
| > What's the first thing that Sydney Goldberg says to Reuven
| after he gets hit in the eye by the baseball?
|
| and ChatGPT responds:
|
| > The first thing Sydney Goldberg says to Reuven after he gets
| hit in the eye by the baseball is, "That was a great catch,
| Reuven! That was sensational!".
|
| Curious thing is, the name is spelled _Sidney_ Goldberg.
| https://archive.org/details/chosen0000chai_y4e8/page/32/mode...
| jeffbee wrote:
| These chatbots just adopt your typos. If I ask Gemini about
| the architecture of _blaze_ instead of _bazel_ it will write
| paragraphs using _blaze_ consistently even though it doesn 't
| exist.
| stavros wrote:
| Blaze is the name of the Google tool that Bazel was based
| on.
| jeffbee wrote:
| I realize that, I was trying to trick it into sending me
| internal documentation. Instead what it does is describe
| all the places I can find information about blaze, such
| as at https://blaze.build ... it just runs with whatever
| you told it.
| stavros wrote:
| You're definitely right that they adopt your typos, and
| that it adopted your typo in that case, I'm just pointing
| out that a tool called Blaze _does_ exist.
| og_kalu wrote:
| Yeah. A few people on X have had access for a couple days now.
| The conclusion is that it's a genuine context window advance, not
| just length, but utilization. It genuinely utilizes long context
| much better than other models. Shame they didn't share what led
| to that.
| emporas wrote:
| >" While Gemini Pro 1.5 is comfortably consuming entire works of
| rationalist doomer fanfiction, GPT-4 Turbo can only accept
| 128,000 tokens."
|
| A.I. Doomers will soon witness their arguments fed into the
| machine, generating counter-arguments automatically for 1000
| books at a time. They will need to incorporate a more and more
| powerful A.I. into their workflow to catch up.
| karmasimida wrote:
| I think the retrieval is still going to be important.
|
| What is not important is RAG. You can retrieval a lot of
| documents in full length, not need to do all these
| chunking/splitting, etc.
| kromem wrote:
| Depth isn't always the right approach though.
|
| Personally, I'm much more excited at the idea of pairing RAG
| with a 1M token context window to have enormous effective
| breadth in a prompt.
|
| For example, you could have RAG grab the relevant parts of
| _every single_ academic paper related to a given line of
| inquiry and provide it into the context to effectively perform
| a live meta-analyses with accurate citation capabilities.
| jeffbee wrote:
| How do people get comfortable assuming that these chat bots have
| not hallucinated? I do not have access to the most advanced
| Gemini model but using the one I do have access to I fed it a
| 110-page PDF of a campaign finance report and asked it to
| identify the 5 largest donors to the candidate committee ...
| basically a task I probably could have done with a normal machine
| vision/OCR approach but I wanted to have a little fun. Gemini
| produced a nice little table with names on the left and aggregate
| sums on the right, where it had simply invented all of the cells.
| None of the names were anywhere in the PDF, all the numbers were
| made up. So what signals do people look for indicating that any
| level of success has been achieved? How does anyone take a large
| result at face value if they can't individually verify every
| aspect of it?
| kromem wrote:
| I'm most excited at what this is going to look like not by
| abandoning RAG but by pairing it with these massive context
| windows.
|
| If you can parse an entire book to identify relevant chunks using
| RAG and can fit an entire book into a context window, that means
| you can fit relevant chunks from an entire reference library into
| the context window too.
|
| And _that_ is very promising.
| zmmmmm wrote:
| The question I would like to know is whether that just leads
| you back to hallucinations. ie: is the avoidance of
| hallucinations intrinsically due to forcing the LLM to consider
| limited context, rather than directing it to specific / on
| topic context. Not sure how well this has been established for
| large context windows?
| patrickhogan1 wrote:
| Lol. So many of these PR posts. Just let real people use the
| thing if it's so good. The more dressed up posts of people having
| "life changing" results without letting people use it increases
| the chance of expectation collapse.
___________________________________________________________________
(page generated 2024-02-23 23:00 UTC)