https://scottaaronson.blog/?p=6288 Shtetl-Optimized The Blog of Scott Aaronson If you take nothing else from this blog: quantum computers won't solve hard problems instantly by just trying all solutions in parallel. Also, next pandemic, let's approve the vaccines faster! --------------------------------------------------------------------- << Scott Aaronson Speculation Grant WINNERS! AlphaCode as a dog speaking mediocre English Tonight, I took the time actually to read DeepMind's AlphaCode paper, and to work through the example contest problems provided, and understand how I would've solved those problems, and how AlphaCode solved them. It is absolutely astounding. Consider, for example, the "n singers" challenge (pages 59-60). To solve this well, you first need to parse a somewhat convoluted English description, discarding the irrelevant fluff about singers, in order to figure out that you're being asked to find a positive integer solution (if it exists) to a linear system whose matrix looks like 1 2 3 4 4 1 2 3 3 4 1 2 2 3 4 1. Next you need to find a trick for solving such a system without Gaussian elimination or the like (I'll leave that as an exercise...). Finally, you need to generate code that implements that trick, correctly handling the wraparound at the edges of the matrix, and breaking and returning "NO" for any of multiple possible reasons why a positive integer solution won't exist. Oh, and also correctly parse the input. Yes, I realize that AlphaCode generates a million candidate programs for each challenge, then discards the vast majority by checking that they don't work on the example data provided, then still has to use clever tricks to choose from among the thousands of candidates remaining. I realize that it was trained on tens of thousands of contest problems and millions of solutions to those problems. I realize that it "only" solves about a third of the contest problems, making it similar to a mediocre human programmer. I realize that it works only in the artificial domain of programming contests, where a complete English problem specification and example inputs and outputs are always provided. Forget all that. Judged against where AI was 20-25 years ago, when I was a student, a dog is now holding meaningful conversations in English. And people are complaining that the dog isn't a very eloquent orator, that it often makes grammatical errors and has to start again, that it took heroic effort to train it, and that it's unclear how much the dog really understands. It's not obvious how you go from solving programming contest problems to conquering the human race or whatever, but I feel pretty confident that we've now entered a world where "programming" will look different. Update: A colleague of mine points out that one million, the number of candidate programs that AlphaCode needs to generate, could be seen as roughly exponential in the number of lines of the generated programs. If so, this suggests a perspective according to which DeepMind has created almost the exact equivalent, in AI code generation, of a non-fault-tolerant quantum computer that's nevertheless competitive on some task (as in the quantum supremacy experiments). I.e., it clearly does something highly nontrivial, but the "signal" is still decreasing exponentially with the number of instructions, necessitating an exponential number of repetitions to extract the signal and imposing a limit on the size of the programs you can scale to. Email, RSSEmail, RSS Follow This entry was posted on Sunday, February 6th, 2022 at 3:12 am and is filed under Nerd Interest, The Fate of Humanity. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site. 45 Responses to "AlphaCode as a dog speaking mediocre English" 1. Florian Says: Comment #1 February 6th, 2022 at 3:50 am Astonishing indeed! Give it another 10 - 15 years and having seen the latest advancements of OpenAI & DeepMind, I'm confident that the vast majority of white collar workers might be replaced as well. 2. Sid Says: Comment #2 February 6th, 2022 at 4:07 am What's amazing to is that no modeling innovation was needed to get this far -- all that was needed was lots of training data + scaling up. 3. Esteban Martinez Says: Comment #3 February 6th, 2022 at 4:26 am Perhaps what it is unclear is how much the programmer understands! I mean, the real question here is if we (humans) will be able clarify deep notions of what it means consciousness and being. What might be striking for eloquent English speakers is that a good program needs no validation from its human peers! Perhaps we are so deep into neoliberalism logic that anything that does not generate immediate winnings is out of the question. 4. Alexander Kruel Says: Comment #4 February 6th, 2022 at 5:42 am Remember that AlphaCode is just a (not fully trained) 41b-parameter model and there are already 280b language models. Scaling laws suggest that larger models can become dramatically more sample-efficient and better at generalization. Also, don't underestimate other ways to improve upon it. Compare e.g. AlphaGo with AlphaGo Zero. It appeared to develop the skills required to beat top humans within just a few days, whereas the earlier AlphaGo took months of training to achieve the same level. 5. Soren Elverlin Says: Comment #5 February 6th, 2022 at 6:01 am >It's not obvious how you go from solving programming contest problems to conquering the human race The standard answer is that this goes through recursive self-improvement. If I was hiring a human to improve AlphaCode, I would consider the skills demonstrated by AlphaCode to be relevant, though not central. 6. Jon Awbrey Says: Comment #6 February 6th, 2022 at 7:02 am Connectionism never learns. Probably why it's such a perfect partner for capitalism. 7. Scott Says: Comment #7 February 6th, 2022 at 7:17 am Jon Awbrey #6: With, I suppose, the more declarative forms of AI that sounded good but never actually worked being a perfect partner for communism? 8. Fast typist Says: Comment #8 February 6th, 2022 at 7:18 am Until AI solves factoring by classical methods I doubt AI supremacy can happen. Scott are you confident AI can find a PTIME factoring algorithm and that there is one? 9. Sid Says: Comment #9 February 6th, 2022 at 7:20 am @Alexander #4 One issue is what wld be the equivalent of self play for program synthesis is a lot less clear 10. J Storrs Hall Says: Comment #10 February 6th, 2022 at 7:23 am During the 1980's it was the practice of the Student Chapter of the ACM at Rutgers to hold annual programming contests. This author had the honor of being one of the judges at such a contest. It worked as follows: The entrants were teams, of from 3 to 5 students. Each team was given an assignment of four programs to write. The team which got all four programs running correctly the earliest, won. Normally most of the students used Pascal, which was the language taught in computer science courses, or Basic, which many had started in. A few engineering students did their programming in Fortran. This year there was entered a motley team of mavericks. Instead of working together, each member would do his work alone, and in a different language. One, who had interned at Bell Labs, would work in C. Another, God help him, was going to use assembly language. A third would try Lisp. And the other member of the "screwball" team, whose name was Damian Osisek, worked in SNOBOL. The contest was unexciting that night, because Mr. Osisek, working alone, completed all four programs before anyone else, individual or team, completed even one. The following year, the use of SNOBOL (and APL ) was banned. 11. Scott Says: Comment #11 February 6th, 2022 at 7:47 am Fast typist #8: Scott are you confident AI can find a PTIME factoring algorithm and that there is one? Of course I'm not confident of either, or of one conditional on the other--what would make you imagine that I was? 12. Scott Says: Comment #12 February 6th, 2022 at 7:49 am J Storrs Hall #10: So then, why isn't the whole world using SNOBOL now? Should it be? Or are there languages in use today (Python? Perl?) with which Damian Osisek would've done just as well? 13. Andrew Kay Says: Comment #13 February 6th, 2022 at 7:50 am Whilst this is an amazing achievement, and obviously just the start, I feel something is missing. Take for example the "backspace" algorithm in the paper. Before I'd let that loose controlling my pacemaker or power plant, or even passed its code review, I'd need at least some sort of mathematical justification why the "start-from-the-end-in-greedy-fashion" actually picks out a correct string of deletions and doesn't have to backtrack over earlier decisions. Maybe in future they'll also generate proof-checking assertions that can be verified formally. Then code review simply has to check that the specification is correct. 14. Scott Says: Comment #14 February 6th, 2022 at 7:59 am Andrew Kay #13: While my programming career was admittedly not long, I never formally verified a single program I wrote! On the other hand, I see little reason why similar methods couldn't be turned loose on the problem of automatically generating formal specifications from English descriptions, if there were a similarly large training corpus. And then one could use the same methods again, and/or leverage existing work, to search for programs that matched the specifications. 15. David Vedvick Says: Comment #15 February 6th, 2022 at 8:02 am Great perspective, thanks for writing this. Now I am wondering, once we have confidence that AI can solve programming problems, how will we master communicating the problem to AI? As a developer, my most common struggle is not solving a hard algorithmic problem, it's determining the problem the customer actually wants solved. 16. Fast typist Says: Comment #16 February 6th, 2022 at 8:08 am How else would we measure AI supremacy? Speeding up mundane tasks humans do? No. It has to be something we know is possible but cannot achieve. I would think you would have assumed factoring is easy because of similarity between Fq[x] and Z and that there is a fast factoring algorithm over Fq[x]. There is no deterministic algorithm over Fq[x]. Perhaps there is an unifying deterministic algorithm which is classical and we do not know how but a smart AI can figure it. 17. Jon Awbrey Says: Comment #17 February 6th, 2022 at 8:08 am Re: Scott Being more triadic than dyadic in my thinking, I've never regarded capitalism and communism -- the latter of which in its collectivist extremes is very connectionist indeed -- to be dichotomous choices, but more like precocious (0.666-baked) approximations to representative democracy. 18. anon Says: Comment #18 February 6th, 2022 at 8:23 am I came up with an algorithm (the same as the AI's) for solving the linear system using a CAS. Feels relevant because this is heralding an era of much more powerful CASes. 19. Scott Says: Comment #19 February 6th, 2022 at 8:27 am David Vedvick #15: I completely agree that figuring out what the customer wants is a huge part of practical software development and likely to be harder for AI. When I told a 9-year-old of my acquaintance about AlphaCode, her first questions were: "but can it write a program to draw a picture of my dad's butt? How would the program fit that butt on the screen?" Imagine translating that into a clear specification! 20. Gerard Says: Comment #20 February 6th, 2022 at 8:51 am Scott. When I first saw the title of your post "AlphaCode as a dog speaking mediocre English", I thought this was going to be a complete takedown of AlphaCode, so I was very surprised at where you ended up. Somehow that title just didn't translate for me into "AlphaCode looks like an amazing advance in AI". I remember when IBM first tried to beat Kasparov. I think I was in college at the time and I didn't believe a computer would ever beat the world champion chess player. Then a few years later it happened and the lesson I Iearned from that was "never bet against the computer" (a lesson that has been quite amply reinforced several times since). Still it's a very weird process. About 10 years ago I watched Watson beat the best Jeopardy contestants and I was deeply impressed. Surely IBM had developed some really powerful technology there. Then a few weeks ago we learned that they sold off the Watson division (which had made a very non-pbvious and , in my opinion, questionable, pivot into healthcare) for a fraction of what they had invested in the technology. PS. Why do you need to solve that system of equations without Gaussian Elimination ? It's quite an easy algorithm to implement, I did it recently in a few lines of Python for some coding challenge. 21. dlb Says: Comment #21 February 6th, 2022 at 8:52 am Long time lurker, first time poster. As a professional programmer I can confirm that a *lot* of my work can be efficiently replaced by good algorithms like this one. When I was young, our job was to stick many bits into expensive memory (so to speak). Today, it is to connect many libraries together to get the job done. In the future, it could very well be to explain the problem at hand to some "AI". Still, understanding how to formulate a problem so that an algorithm like AlphaCode can solve it looks like a skill in itself to me. So programmers have a future, just different than what we know today (and hopefully, with the same number of programmers just solving a larger amount of problems, or big problems a single programmer cannot solve alone). As for an AI taking over the world, chess players, go players and now programmers haven't been able to do it. I don't see how the AI could. PS: your blog is the best - if one ignores US-only-the-sky-is-falling posts, but even with the noise the signal is excellent -. Keep the good work! 22. Boaz Barak Says: Comment #22 February 6th, 2022 at 9:15 am Agree it's super impressive, but maybe the analogy is a dog that can speak in extremely eloquent paragraphs, making fewer grammatical (the analogs of "off by one") mistakes than most humans, but its ability to reason over a longer time scale is not much better than current dogs. 23. Danylo Says: Comment #23 February 6th, 2022 at 9:16 am > but I feel pretty confident that we've now entered a world where "programming" will look different The question is - what will be the meaning of the word "we" in the future By the way, I find OpenAI announcement about solving math olympiad problems even more striking, because it's much closer to how humans actually think - by using a neural network to find and combine formal rules of logic. 24. Scott Says: Comment #24 February 6th, 2022 at 9:35 am Gerard #20: PS. Why do you need to solve that system of equations without Gaussian Elimination ? It's quite an easy algorithm to implement, I did it recently in a few lines of Python for some coding challenge. I was careful to say, you need to avoid Gaussian elimination in order to solve it "well." Gaussian elimination would be an ugly cubic-time solution where a nice linear-time solution exists--and AlphaCode indeed finds the latter. It's an interesting question whether AlphaCode could also have found the ugly solution--probably it depends on whether or not Gaussian elimination was common in its training corpus? As for the ease of implementation: YMMV! One of my most-cited papers, Improved Simulation of Stabilizer Circuits (with Daniel Gottesman), came about because as a grad student doing a course project 19 years ago, I didn't feel like implementing Gaussian elimination in C and didn't know how to call it in a library. So instead I spent a few days searching for a better solution until I found one, and then Daniel explained to me why it worked. 25. Scott Says: Comment #25 February 6th, 2022 at 9:40 am Danylo #23: By the way, I find OpenAI announcement about solving math olympiad problems even more striking, because it's much closer to how humans actually think - by using a neural network to find and combine formal rules of logic. That was indeed also striking! Unless I'm missing something, though, a crucial difference between the two is that the OpenAI thing takes as input a hand-coded formal specification of the IMO problem, then uses deep learning to help search for a formal proof. AlphaCode takes as input the plain English (!!) specification of the programming competition problem. 26. Gadi Says: Comment #26 February 6th, 2022 at 9:40 am I think people underestimate how hard it is finding the correct question to result in general intelligence. Just think about the hundreds of millions of years of evolution and the uncountable number of somewhat-intelligent yet not quite there animals that lived in this planet. Yet all this time none of them became as intelligent as humans, and it's not even quite clear what made humans make the leap. Maybe "how to program from an English description" will be another miracle question leading to intelligence, but I don't think it's harder than the kind of questions evolution had been solving for millions of years. We're not using "better technology" than nature with artificial neural networks, if anything, the neural networks we're using are orders of magnitudes weaker. Yes, we know this technology eventually created human intelligence - but only after hundreds of millions of years. Either way, dogs understand mediocre English. Many animals do, and only lack vocal cords to speak back. Parrots can even speak back. Monkeys can speak sign language. Maybe the one in a trillion event was the invention of language itself, but other animals also developed all sorts of languages yet didn't reach human intelligence. Whatever is the question to which the answer is human intelligence, it had to be pretty non-trivial for us to be the first intelligent species. And that's assuming it was one question and not a strange combination of questions that together resulted in this miracle. I'm definitely not afraid of AI becoming the new overlords from research about computer vision, or about winning games. Nature was pitting neural networks against neural networks in this game for millions of years until it reached us. Just think about the immense computational power that required reaching humans in the first place, you'd have to have a very good reason to believe you have a shortcut to all of that computation. My bet is that the first general intelligence will come from just simulating a human brain, using all that computation time of nature instead of trying to invent humans from scratch. 27. Scott Says: Comment #27 February 6th, 2022 at 9:46 am dlb #21: PS: your blog is the best - if one ignores US-only-the-sky-is-falling posts, but even with the noise the signal is excellent -. Keep the good work! Thanks!! Regarding the US, though, the thing is that the sky did fall. We're now a decrepit, crumbling empire, torn between fanatically self-certain factions, that can no longer maintain the ideals of freedom, democracy, progress, and Enlightenment at home let alone export them to the rest of the world. Is the problem just that I was ever naive enough to expect anything different? 28. Ivo Says: Comment #28 February 6th, 2022 at 9:49 am Scott, you might enjoy this mind blowing paper on combining GPT-3 with Codex to plan and execute actions for embedded agents: https://twitter.com/pathak2206/status/1483835288065658882?s=20&t= sBfBzd8l7kwtLWm-Ptderg The agents are in VR for now, but one can see we are rapidly moving coser to a world where all intellectual and physical work is done by AI and not us. 29. A Raybold Says: Comment #29 February 6th, 2022 at 10:05 am Scott #14: Could this or similar methods be effective in generating formal specifications from English descriptions? I don't doubt that a similar method could generate a number of candidate formalizations of the problem, but how would a likely-correct one be selected? In the case of AlphaCode, example solutions are essential, and coming up with examples to work with is beyond its abilities. An AlphaFormalMethods program that similarly depends on an input of examples would be missing the same essential step as is also missing from AlphaCode. If your point here is that formal verification is a red herring here, then I would agree, on the basis of what I have written above, but I think Andrew Kay #13 still has a point: AlphaCode is not demonstrating any of the judgement that humans put to use on this sort of task - on the contrary, it is effectively being outsourced to humans. When you consider where we go from here, I suspect the question of judgement will loom large (if it is not already an issue for things like self-driving cars.) Maybe current machine learning methods can deliver on that too - most of us (myself included) have been surprised already. 30. Gerard Says: Comment #30 February 6th, 2022 at 10:08 am Gadi #26 > Maybe the one in a trillion event was the invention of language itself That would be my bet. Natural language in the human sense appears to be a completely general method of representing information. Formal languages used in logic, computer science and mathematics are really just subsets of natural language and as I understand it the entire theory of logic and computer science (at least, mathematics I'm less sure of) can be formulated in terms of transformations between different formal languages (ie. again, just subsets of natural language). For that reason I think I would expect the first signs of real progress towards AGI to appear in NLP-like systems. 31. J Storrs Hall Says: Comment #31 February 6th, 2022 at 10:12 am Scott # 12: If all the programmers in the world were at Osisek's level, we would be using something far beyond SNOBOL (modern equivalent, Prolog) unified with, say, Matlab. The key question is how many higher level concepts the programmer is able to think in, and only then whether the language supports them directly. I imagine that say Python with all its libraries more than covers the range in which the typical programmer is able to think. Anyone who digested the singers problem into a linear system as you did above would be able to write a solution in one line of a modern APL. But I fear that leaves out a majority of practicing programmers. 32. Scott Says: Comment #32 February 6th, 2022 at 10:13 am A Raybold #29: Oh, I was imagining that the AI to generate the formal spec would also take some examples as input. If "programming" can be reduced to "provide some examples of the input/output behavior you want, plus some English text to prime an AI in the right direction to generate a formal spec and/or the actual code," do people have any idea what a huge deal that is? 33. Jeff Lun Says: Comment #33 February 6th, 2022 at 10:16 am I think the issue with most of the critics of this kind of progress is that most of the arguments against assume: 1. The the critic themselves thinks of themselves as an above-average programmer, which may or may not be true 2. It's easy to point out and focus on failures rather than progress (I think this blog article does a good job of pointing of the progress side) If you think of systems like AlphaCode as an attempt to approach the problem more from a statistical standpoint, where the goal is not to write code like a human would, but instead to write code at lower cost in aggregate over time, it starts to look a lot more like how the insurance industry works. For example, when seatbelts and airbags were introduced to cars vs. when they became mandatory has a similar story arc. At first people would say, "Why do you need these safety devices - just drive better!" or "Well, seatbelts save some lives, but there are situations where they caused death that wouldn't have happened." The problem with both of these statements is that they seek to point out the lack of perfection in specific cases, whereas the insurance industry is more interested in seeing overall improvement on the aggregate. The proxy in the insurance world (if I understand it approximately correctly) is something like: "cars are getting safer if the total dollar amount of claims paid out per mile driven is reduced over time." There are a million possible ways to reduce claims (including outright rejecting them), but in the idealized case where everyone's an honest broker, the principle is that what you're optimizing for isn't the complete elimination of all death, but a reduction in the cost, frequency, and severity of injuries, deaths, and property damage. To bring this back to things like AlphaCode: so long as AlphaCode (or any alternative implementation, for that matter) can reduce the cost of producing a system that produces a correct mapping from inputs to outputs for any given problem definition, then you've achieving the goal. In other words, I don't care if AlphaCode has to try 1,000,000 candidate solutions. What I care about is: given a set of test cases (like unit tests), can the computer come up with a valid solution in less time, and at less total energy cost than a human? If so, then the cost to produce correct solutions to programming problems has been reduced, and whatever that new thing that has reduced the cost - that thing can be turned into a tool that is given to human programmers, thereby making them more effective. Rinse and repeat and you may not get computers replacing programmers anytime soon, but you may very well increase the efficiency (in terms of time and cost) of programmers by 10%-30%, and that along is worth billions of dollars per year. Now take that 10%-30% efficiency gain, COMPOUND it over several years, and across an entire industry and you're talking about generations worth of productivity gains in just a few nears - and theoretically it builds on itself. There may be some asymtotic efficiency maximum somewhere, but if so I guarantee we still have plenty of efficiency to gain. 34. Akshat Says: Comment #34 February 6th, 2022 at 10:18 am > And people are complaining that the dog isn't a very eloquent orator, that it often makes grammatical errors and has to start again, that it took heroic effort to train it, and that it's unclear how much the dog really understands But these points all correctly undermine the impressiveness of the effort: they indicate that we are blindly exploring the terrain of training dogs to speak English, rather than that we actually understand how we're doing it. This kind of demonstration is illusory progress. With a great deal of money and effort, we have succeeded in constructing a Rube Goldberg machine. We can make no guarantees about how this Rube Goldberg machine would perform outside of our carefully curated habitat. We can't tell you how it really works. If you're lucky, a third of the time, it will do what we wanted it to do. Even more damningly: in principle, nothing prevented us from doing it twenty years ago, except there wasn't enough hardware and nobody wanted to spend that much money. Procurement of additional funding is not really evidence of foundational progress. Foundational progress comes from explainability, which in turn leads to iteration. When there exists an accessible model for how input becomes output that we can refine, that's when we should sound the victory bells. 35. A Raybold Says: Comment #35 February 6th, 2022 at 10:29 am Scott #31 I see what you mean - it's like saying that if you take a planetful of matter orbiting a star, and just leave it alone for several billion years, then, simply by that matter interacting under the constraints of physics, you might end up with a creature that figures out this is what happened - mind-blowing stuff! The key to it all is having something - survival of the fittest, a collection of right answers - to pick out the winners. 36. Scott Says: Comment #36 February 6th, 2022 at 10:32 am Akshat #34: See, that's where you're wrong. With the use of deep learning for translation, voice recognition, and face recognition, we similarly don't understand in any detail how it works, yet those applications have changed the world. AlphaCode looks to me like it could already be useful for practical programming, just as it stands, and of course we should expect such things to improve dramatically in the coming years. This would likely have cost billions of dollars to do 20 years ago, if it was possible at all. You can say that that's "merely" down to Moore's Law, plus the existence now of gargantuan programming competition datasets, plus all the practical deep learning experience of the past decade, but if so, the word "merely" is doing a huge amount of work! That more compute and more training data beats every attempt to hardcode in "semantic meaning" is, indeed, the Bitter Lesson of AI. And as a scientist, I'm committed to the idea that, when reality tries as hard as it possibly can to teach us a certain lesson, it's our responsibility to learn the lesson rather than inventing clever reasons to maintain the contrary worldview that we had before. 37. Gerard Says: Comment #37 February 6th, 2022 at 10:56 am Scott #36 > With the use of deep learning for translation, voice recognition, and face recognition, we similarly don't understand in any detail how it works, yet those applications have changed the world. How much have they really changed it though ? I'll grant you that the fact that today anyone can typically get a reasonably good gist of a foreign language publication just from Google Translate is a pretty big deal, but I'd put it on a similar or lower level of "changed the world" as smartphones, the Internet or social media, rather than say internal combustion engines, electricity or H-bombs. Still no one would seriously want to use machine translation for a mainstream media article, let alone anything really important like a legal document. I guess lots of people are using voice translation for simple queries to virtual assistants but I don't think many doctors or lawyers are using it for automatic dictation. Or, if they are, they're sure to carefully proofread the resulting product. As for face recognition, it's something that gets talked about a lot in the press, but I'm skeptical that the coverage accurately reflects reality. I don't think we're anywhere near having a system that can watch Times Square and put a name on virtually every person who walks by. In fact I doubt such a thing is possible because in a crowded space even with lots of cameras you aren't likely to get many unobstructed pixels on any particular face (of course that's without even considering the problem that these days many people will be wearing masks). 38. Veedrac Says: Comment #38 February 6th, 2022 at 11:09 am Scott #34: And as a scientist, I'm committed to the idea that, when reality tries as hard as it possibly can to teach us a certain lesson, it's our responsibility to learn the lesson rather than inventing clever reasons to maintain the contrary worldview that we had before. Excellently put. As a further quiz to those that would rather not see; If machine learning is not at all a general understander, how do the same neural circuits running the same algorithms apply to almost any domain, whether language or image synthesis or playing Go or theorem proving? If learned models do not at all implement general cognition, why does pretraining on language markedly improve non-language tasks, like reinforcement learning on Atari games? If neural networks are entirely oriented around memorization of previously seen ideas, why do they contain semantically meaningful features we did not train on, like how does AlphaGo know to calculate liveness? In fact, how is a model meant to know how to interpolate data points at all in high dimensional spaces without some semantically-meaningful understanding of that space? We tried naive methods for language modelling before, they were called Markov Chains, and they could rarely produce coherent sentences, never mind be offered a new word and use it in a sentence. The reality is that we're in a world where iGPT can complete the top half of this image as this completion, and the best argument the naysayers have as to how it made a semantically meaningful completion amounts to "probably it saw that completion it made before, with a different texture." But it didn't. There are not enough images in the world. 39. Nick Nolan Says: Comment #39 February 6th, 2022 at 11:12 am it clearly does something highly nontrivial, but the "signal" is still decreasing exponentially with the number of instructions, necessitating an exponential number of repetitions to extract the signal and imposing a limit on the size of the programs you can scale to. This describes human intelligence quite well. Our domain is short problems that can be modeled spatiotemporally in 3d-mostly picking berries and avoiding predators. When we try to extend ourselves to other tasks (modern physics, mathematics, logic) the number of repetitions, false starts to grow exponentially. We have limit for the size of the problems we can solve fast, then comes exponential slowdown. Group intelligence and language have rescued us from noticing our limits it because we stand on each others shoulders. We spend lots of time making things simpler and condensing problems into short rules we can follow. 40. Gerard Says: Comment #40 February 6th, 2022 at 11:32 am Scott: > A colleague of mine points out that one million, the number of candidate programs that AlphaCode needs to generate, could be seen as roughly exponential in the length of the generated programs. I don't understand that remark. Surely the programs are far longer than 20 bits in length. 41. Gadi Says: Comment #41 February 6th, 2022 at 11:42 am Gerard #30: I don't buy it. Nature did evolve communication similar to language in many other species. Why did only humans get this far? If it's just the invention of language and brains were just fit for using it from the moment it was invented, then we could have taught other species to be as intelligent as us just by teaching them language. But we can't. We've been talking to animals for a long time and they didn't become smarter, even after thousands of years of evolutionary pressure from domestication. My bet is that's it's not just language. Even if it is language and it co-evolved with human brains, I'm not sure you can replicate whatever process happened over thousands of years and billions of humans with computations several orders of magnitude smaller. Maybe at best you can mimic human intelligence by getting inputs from humans, but creating something smarter or even equal, on its own? 42. Scott Says: Comment #42 February 6th, 2022 at 11:47 am Incidentally, Gerard #20: When I first saw the title of your post "AlphaCode as a dog speaking mediocre English", I thought this was going to be a complete takedown of AlphaCode, so I was very surprised at where you ended up. Ah, but that's exactly the point, isn't it? People's instinct to sneer at AlphaCode is wrong in exactly the same way as their instinct to sneer at a haltingly talking dog would be. 43. Danylo Says: Comment #43 February 6th, 2022 at 1:06 pm Scott #25 Yes, but transforming a language description of a problem into a formal form and solving it in a formal form are two different tasks. And it's reasonable to solve them separately. It's amazing that AlphaCode can do it using a unified approach. But I don't think it's the correct way to proceed further. General AI has to invent formal laws one way or another. With our help or without. But if AI will invent, say, ZFC, but without our help, then it will be a complete blackbox to us. It will be much harder to communicate. There is already a huge problem with explanation of ML models outputs. Yet, they are used in criminal justice, healthcare and other domains that affect human lives directly (see, e.g., https://arxiv.org/abs/1811.10154). 44. drm Says: Comment #44 February 6th, 2022 at 2:29 pm Its cousin AlfphaFold2 is likely to win a Nobel for solving the protein-folding problem for most (but not all) practical purposes. Its done a huge heavy lift for biology, delivering 100's of thousands of structures to the community in less than a year. (Of course if it turns out their not accurate after all, then it will be one of great head fakes in the history of science:) I have run it locally, its a fascinating machine. 45. Arthur Says: Comment #45 February 6th, 2022 at 3:35 pm I feel like this is a good example of the fallacy pointed out by Rodney Brooks in http://rodneybrooks.com/ the-seven-deadly-sins-of-predicting-the-future-of-ai/, specifically in the section of "performance vs. competence". The image of a dog speaking mediocre english feels fantastic in part because we would expect the dog to be able to tell us about its point of view, explain how it spent its day, etc. We are carrying all the baggage of our expectations of "dog" with the idea of "dog that can kind of speak english". The work behind alphacode is imo fantastic; but it has to be viewed through the proper lens. I have played with a lot of these kinds of models (large language models finetuned for code, or even large models trained on github dumps), and if you fall just a tiny bit off distribution you get craziness. It's also not obvious how many of the solutions to the problems are at least part in the training data. I don't wish to minimize how cool I think this work is, its really great. but its not anything like a talking dog. Leave a Reply Comment Policy: All comments are placed in moderation and reviewed prior to appearing. Comments can be left in moderation for any reason, but in particular, for ad-hominem attacks, hatred of groups of people, or snide and patronizing tone. Also: comments that link to a paper or article and, in effect, challenge me to respond to it are at severe risk of being left in moderation, as such comments place demands on my time that I can no longer meet. You'll have a much better chance of a response from me if you formulate your own argument here, rather than outsourcing the job to someone else. I sometimes accidentally miss perfectly reasonable comments in the moderation queue, or they get caught in the spam filter. If you feel this may have been the case with your comment, shoot me an email. You can now use rich HTML in comments! You can also use basic TeX, by enclosing it within $$ $$ for displayed equations or \( \) for inline equations. [ ] Name (required) [ ] Mail (will not be published) (required) [ ] Website [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [Submit Comment] [ ] [ ] [ ] [ ] [ ] [ ] [ ] D[ ] --------------------------------------------------------------------- Shtetl-Optimized is proudly powered by WordPress Entries (RSS) and Comments (RSS).