https://garymarcus.substack.com/p/about-that-openai-breakthrough Marcus on AI Subscribe Sign in Share this post [https] About that OpenAI "breakthrough" garymarcus.substack.com Copy link Facebook Email Note Other [https] Discover more from Marcus on AI "Amongst the myriad of opinions, one voice stands out as a rational and impartial advocate: Gary Marcus." --Candice Clark Over 28,000 subscribers [ ] Subscribe Continue reading Sign in About that OpenAI "breakthrough" Will OpenAI's Q* change the world? [https] Gary Marcus Nov 23, 2023 49 Share this post [https] About that OpenAI "breakthrough" garymarcus.substack.com Copy link Facebook Email Note Other 32 Share [Newscaster]: We interrupt your Thanksgiving to bring you this bulletin. Yesterday at The Information, in the continuing drama surrounding OpenAI: [https] Later, at Reuters, further developments: [https] The technique in question appears to be something OpenAI is calling both ominously, as Q*, pronounced "Q-Star." Get it? Perfect grist for anyone who wants to freak about the Death Star killing us all. (In reality, though, it seems to be named in reference to a clever but not particularly ominous AI technique known as A* that is often used to power character movements in video games..) The board may well have genuinely have been worried about the new techniques. (It's also likely they had other stuff on their mind; according to the same story in Reuters, the handling of Q* is allegedly "[just] one factor among a longer list of grievances by the board leading to Altman's firing, among which were concerns over commercializing advances before understanding the consequences".) SS Me? I am pretty skeptical; I was only half-kidding when I wrote this: [https] SS Part of the joke is that I don't actually think Bing is powered already by the alleged breakthrough. A single fail on Bing a few months after said putative breakthrough isn't going to tell us much; even if it were real, whipping Q* (or any other algorithm) it into production that fast would be extraordinary. Although there is some talk of OpenAI already trying to test the new technique out, it wouldn't be realistic for them to have entirely changed the world in a matter of months. We can't really take Bing 2023 to tell us anything about what Q* might do in 2024 or 2025. But then again I have seen this movie before, often. OpenAI could in fact have a breakthrough that fundamentally changes the world. But "breakthroughs" rarely turn to be general to live up to initial rosy expectations. Often advances work in some contexts, not otherwise. Arguably every putative breakthrough in driverless cars has been of this sort; somebody finds something new, it looks good at first, maybe even helps a bit, but at the end of the day, "autonomous" cars still aren't reliable enough for prime time; no breakthrough has gotten us over the threshhold. AV's still aren't general enough that you can just plop down a car that was tuned in pilot studies in Menlo Park, SF and Arizona and expect it to drive gracefully and safely in Sicily or Mumbai. We are probably still many "breakthroughs" away from true level 5 driverless cars. SS Or ronsider what was touted at OpenAI as an extraordinary breakthrough in 2019, when they launched a slick video and blog post about how they'd gotten a robot to solve a Rubik's cube:: [https] Link in the screenshot above isn't live, but you can see it at https: //openai.com/research/solving-rubiks-cube.. To many, the result sounded amazing. VentureBeat gullibly reported OpenAI's PR pitch wholesale; "OpenAI -- the San Francisco-based AI research firm cofounded by Elon Musk and others, with backing from luminaries like LinkedIn cofounder Reid Hoffman and former Y Combinator president Sam Altman -- says it's on the cusp of solving something of a grand challenge in robotics and AI systems." Me being me, I called bullshit, slamming OpenAI on Twitter a few days later: [https] Probably the most relevant bullet point on the right (at least for present purposes) was the one about generalization; getting their algorithm to work for one object (which, cheesily, turned out to be a special Cube instrumented with sensors and LEDS, unlike a Rubik's cube you'd buy in a store) in carefully controlled lab circumstances hardly guaranteed that the solution would work more broadly in the complex open-ended real world. Notwithstanding my concerns, OpenAI got a tremendous amount of press, and maybe some investment or recruiting or both off the press release. Looked good at the time. But, you know what? It went nowhere. A year or two later, they quietly closed down the robotics division down. SS The thing that cracked me up the most about the Reuter's piece was the wild extrapolation at the end of this passage: [https] If I had a nickel for every extrapolation like that--today , it works for grade school students! next year, it will take over the world! --I'd be Musk-level rich. SS All that said, I am scientist. The fact that a lot of past performance has been overhyped no guarantee that every future advance will fail to pan out. Sometimes you get nonsense about room-temperature superconductors (such narratives always play well initially), and some things really do pan out. It's all still an empirical question; I for one certainly don't yet know enough about the details of Q* to judge with certainty. Time will tell. But, as for me, well... I got 99 problems I'm worried about (most especially the potential collapse of the EU AI Act, and the likely effects of AI-generated disinformation on the 2024 elections). At least so far, Q* ain't one. Share Gary Marcus has been resisting hype for three decades; only very rarely has he been wrong. Marcus on AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. [ ] Subscribe 49 Share this post [https] About that OpenAI "breakthrough" garymarcus.substack.com Copy link Facebook Email Note Other 32 Share 32 Comments [https] [ ] Share this discussion [https] About that OpenAI "breakthrough" garymarcus.substack.com Copy link Facebook Email Note Other Aaron Turner 4 hrs ago*edited 4 hrs agoLiked by Gary Marcus From an ML perspective, LLMs are an amazing achievement. Who would have expected such impressive (from the perspective of an external observer) performance from such a simple NN model trained with vast amounts of (relatively low-quality) data, and vast amounts of compute? From an AGI perspective, however, if your objective was to build literally the *worst* AGI possible (relatively weak [https] cognition (lots of data plus statistical induction plus analogical reasoning, but no deduction or abduction), extremely shallow (if any) internal model of the physical universe, appallingly misaligned with aggregate human preferences, no understanding of how it actually works inside, let alone mathematically well-founded), then you couldn't have done a better job. I await further details of Q* with trepidation... Expand full comment Reply Share Nicholas LeCompte Writes Dumber Than A Honeybee 4 hrs agoLiked by Gary Marcus I am sure Q* makes ChatGPT a better consumer product and that the paper is worth reading. But the stuff about "passing math on the level of grade school students" once again fails to acknowledge that GPT-4 fails an infinite number of math tests designed for *rats* - are there more walnuts or more acorns? what if I add peanuts to the mix? - press this lever four times if you want a cracker GPT-4 can be trained to solve all sorts of specific math [https] problems but it never actually figured out what a number is and therefore lacks the most basic quantitative reasoning. If GPT-4 were an animal, it could count acorns, but bizarrely not peanuts. The acorn-specific tricks around counting do not extend to an *abstract understanding* of numeracy. I just don't think OpenAI's result could possibly be as meaningful as they say, since they don't acknowledge that core numeracy is a problem. Human-language "'math problems" are an algorithmic means to an end, the quantitative reasoning is the actual thought. There's no point in talking about "AGI" if OpenAI continues to be lazy and cynical about the difference between a mathematician and a computer algebra system. Expand full comment Reply Share 10 replies 30 more comments... Top New Community No posts Ready for more? [ ] Subscribe (c) 2023 Gary Marcus Privacy [?] Terms [?] Collection notice Start WritingGet the app Substack is the home for great writing This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts