[HN Gopher] Peter Norvig critically reviews AlphaCode's code qua...
___________________________________________________________________
Peter Norvig critically reviews AlphaCode's code quality
Author : wrycoder
Score : 551 points
Date : 2022-12-16 20:38 UTC (1 days ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| bombcar wrote:
| _The marvel is not that the bear dances well, but that the bear
| dances at all._
|
| The surprising thing is that it can make code that works -
| however, given that code can be _tested_ in ways that "art" and
| "text" cannot (yet), perhaps it's not that strange.
| dekhn wrote:
| I read this as a very well written feature request to the
| AlphaCode engineers (or anybody working on this problem).
|
| I really like Peter's writing style. It's fairly clear, and
| understating, while also making it quite clear there are areas
| for improvement in reach. For those who haven't read it, Peter
| also wrote this gem: https://norvig.com/chomsky.html which is an
| earlier comment about natural language processing, and
| https://static.googleusercontent.com/media/research.google.c...
| which is a play on Wigner's "Unreasonable Effectiveness of
| Mathematics in the Natural Sciences".
| khiqxj wrote:
| > single letter variables
|
| i hate when people put descriptive names on variables that dont
| need it. when im tired i end up reading the name over and over as
| if it has meaning (it doesnt), and subconsciously ponder the
| meaning for up to a minute before snapping out of it.
|
| they actually dont even serve a purpose. its because we are
| reading code in text form that they are needed. conceptually,
| most variables are just something flowing out of one function to
| the next which would be easily repesented by lines in a diagram
| or many other ways that dont use names. you're meant to only give
| names to the rare case where it would truly make the code more
| understandable. this shit doesnt: thingManagerProxyGenerator =
| createThingManagerProxyGenerator()
| krzat wrote:
| This could be a cool task for AI: hover on variable and have
| it's role at a higher level than just finding all references.
| preommr wrote:
| I usually don't like commenting on people's choices for
| variable names since they're usually scoped and less important
| than the interface. However, I'll give my two cents because I
| remember stuff like this being confusing when I was younger.
|
| There's definitely a balance to be struck - one letter variable
| names are horrible outside of single lines (e.g. a one-line
| anonymous function), extremely long names are better, but still
| bad. I say better because it's higher in comprehensibility -
| which is more important - if not very readable. While it's been
| well documented that shorter information is easier to remember,
| it's important to remember that it's usually limited to brevity
| that still provides context. Very short names e.g. (pg for
| proxy generator) are bad because minds require an additional
| step to "unravel" the meaning.
|
| IMO it's usually best to limit to variable names to one or two
| words. Something like 'proxyGenerator', for the given example.
| Or even 'generator' if the parent block is extremely small
| (e.g. 5 lines). Most readers perform word chunking such that
| something like 'proxyGenerator' reads as two units without the
| indirection that an abbreviation or unrelated symbol would
| require.
| nvrmnd wrote:
| This is also a very interesting concept/question when it comes
| to AI writing software for practical applications.
|
| Would a large language model benefit from descriptive variable
| names when it comes to code understanding, or would this just
| be for human benefit? My first thought was that a computer
| (e.g. compiler) does not care what the token is named. But a
| large language model may actually benefit from it, certainly
| chatGPT is sensitive to it.
|
| But then, doesn't it mean that it could be fooled by misleading
| variable names (as a human would), something we would criticize
| any system for. "Alpha*** is easily fooled by changing variable
| names, making it completely unusable for blah blah".
| xwdv wrote:
| In its best form, I think AI coders in our career's lifetime will
| never be anything more than a way to copy and paste code from
| StackOverflow with less steps. If you don't mind searching for
| solutions and adapting them yourself, then you can already be
| living in the future, today!
| MoSattler wrote:
| I wasn't aware AI can already take plain English text and create
| functioning software.
|
| I guess it's time to look for another profession.
| ResearchCode wrote:
| What other profession? When and if it can actually write code,
| making a ppt or sitting in a pointless meeting is a long solved
| problem.
| [deleted]
| weatherlite wrote:
| All the low status professions we've been told are not that
| good - child care, elderly care, handy work etc ...
|
| Yeah we're pretty much toast :)
| WheelsAtLarge wrote:
| Code quality will be an issue since these apps are using existing
| code as the model. They are the very definition of trash in,
| trash out.
| tuke wrote:
| No comment along the lines of: "ew, Python."
| tareqak wrote:
| When I saw the test suite that Peter Norvig created for the
| program, I immediately thought to myself "what if there was a LLM
| program that knew how to generate test cases for arbitrary
| functions?"
|
| I think a tool like that even in an early incomplete and
| imperfect form could help out a lot of people. The first version
| could take all available test cases as training data. The second
| one could instead have a curated list of test cases that pass
| some bar.
|
| Update: I thought of a second idea also based on Peter Norvig's
| observation: what about an LLM program that adds documentation /
| comments to the code without changing the code itself? I know
| that it is a lot easier for me to proofread writing that I have
| not seen before, so it would help me. Maybe a version would
| simply allow for selecting which blocks of code need commenting
| based on lines selected in an IDE?
| Buttons840 wrote:
| How about the other way. I define a few test cases and the AI
| writes code for a generalized solution. Not just code that
| regurgitates the test cases, but that generalizes well to
| unseen cases. You'll notice this is simply the machine learning
| problem restated.
|
| The next step could be to have the AI write code that describes
| its own reasoning, balancing length of code and precision.
| avmich wrote:
| > How about the other way. I define a few test cases and the
| AI writes code for a generalized solution. Not just code that
| regurgitates the test cases, but that generalizes well to
| unseen cases. You'll notice this is simply the machine
| learning problem restated.
|
| I think Barliman does something similar
| (http://minikanren.org/)
| spawarotti wrote:
| > I define a few test cases and the AI writes code for a
| generalized solution
|
| How about the AI never writing any code, just training "mini
| AI" / network that implements the test cases, of course in a
| generalized way, the way our current AI systems work. We
| could continue adding test cases for corner cases until the
| "mini AI" is so good that we no longer can come up with a
| test case that trips it over.
|
| In such future, the skill of being comprehensive tester would
| be everything, and the only code written by humans would be
| the test cases.
| kubb wrote:
| That's potentially more helpful than writing the code itself.
| Writing unit tests can take most of the development time.
| mrguyorama wrote:
| And literally throwing random half junk unit tests at your
| code will better test it than you writing unit tests that are
| blind to the problems it might have because you wrote both
| and both bits of code have the same blind spots.
|
| We should probably be developing systems that fuzz all code
| by default.
| SlickNixon wrote:
| In the future, code beyond all human comprehension will be
| rated for reliability by the number of layers of garbage
| unit tests it's buried under. Tests for the code, then
| tests for the tests, then tests for those te-...
| danuker wrote:
| I warmly recommend QuickCheck-inspired libraries.
| cratermoon wrote:
| Oh yes, AlphaCode could definitely replace some of the clowns
| I've worked with.
| happyopossum wrote:
| > I find it problematic that AlphaCode dredges up relevant code
| fragments from its training data, without fully understanding the
| reasoning for the fragments.
|
| As a non-programmer who has to 'code' occasionally, this is
| literally what I do, but it takes me hours or days to hammer out
| a few hundred lines of crap python. Using a generative model or
| llm that can write equally crappy scripts in seconds feels like a
| HUUUGE win for my use cases.
| peteradio wrote:
| A lazy ineffective person is preferable over a prodigious
| idiot.
| lioeters wrote:
| Reminds me of the quote:
|
| > There are four types of officer: the clever and lazy, the
| clever and industrious, the stupid and lazy, and the stupid
| and industrious.
|
| > The clever and lazy you make Chief of Staff, because he
| will not try to do everybody else's work, and will always
| have time to think.
|
| > The clever and industrious you make his deputy.
|
| > The stupid and lazy you put into a line battalion, and kick
| him into doing a job of work.
|
| > The stupid and industrious you must get rid of at once,
| because he is a national danger.
| happyopossum wrote:
| Not sure which one you're calling me
|
| The reality is that there's a huge gap of scripting needed in
| todays world that sits between "print 'hello world'" and
| something worth paying a professional SWE to write.
|
| You folks don't always see this from inside your bubble, but
| there is a world of automation and computing problems that
| are solved by people who _can_ make it work, and not by
| people _paid_ to make it work well.
| gfd wrote:
| Isn't `a=a[2:]` going to copy and make that linear time too? I'm
| surprised he caught the `b.pop(0)` bug but then suggested doing
| that afterwards.
|
| Also he should submit to codeforces to get a comparable running
| time. For example here is a random person who submitted
| AlphaCode's version and it takes 1091 ms
| https://codeforces.com/contest/1553/submission/144971343
| PartiallyTyped wrote:
| > Isn't `a=a[2:]` going to copy and make that linear time too?
|
| Afaik it's a memcpy of pointers.
| SekstiNi wrote:
| That would indeed be linear time, but he doesn't make that
| suggestion as far as I can see.
| gfd wrote:
| It's in the red inked code review image and it's in his
| rewrite as "source = source[:-2]"
| lisper wrote:
| Is it just me, or is that problem description completely
| incoherent?
| drexlspivey wrote:
| Problem: You open a terminal and type the string 'ababa' but
| you are free to replace any button presses with backspace. Is
| there a combination where the terminal reads `ba` at the end?
| lisper wrote:
| Thanks, your version makes a lot more sense.
| hoten wrote:
| If the AI could do this simplification just as you did, I'd
| find that far more exciting!
| krackers wrote:
| It took me way too long to understand it as well. And the fact
| that you press backspace _instead_ of a character, instead of
| allowing backspace to be pressed at any time (which would turn
| it into checking if B is a subsequence of A I believe).
| zug_zug wrote:
| Good god I'm not alone. For me it's fascinating that an AI can
| make sense of that garble of words. I spent 4 minutes trying to
| read it and gave up.
| [deleted]
| aidenn0 wrote:
| The Minerva geometry answer looks like something one of my kids
| would have written: guess the answer then write a bunch of mathy-
| sounding gobbledygook as the "reasoning."
|
| Also, that answer would have gotten 4/5 points at the local high-
| school.
| fergal_reid wrote:
| Huge respect for Norvig, but I think this is a shallow analysis.
|
| For example, I just took Norvig's 'backspacer alpha' function and
| asked ChatGPT about it. It gave me an ok English language
| description. It names the variables more descriptively on
| command.
|
| I'm sure it'll hallucinate and make errors, but I think we're all
| still learning about how to get the capabilities we want out of
| these models. I wouldn't rush to judgement about what they can
| and can't do based on what they did; shallow analysis can mislead
| both optimistically and pessimistically at the moment!
| neontomo wrote:
| You are talking about a completely different language model
| though, so your argument doesn't really hold. The point he
| makes is that AlphaCode does not generate meaningful
| explanations along with its code (and that's it's perhaps
| necessary to build trust in a system), not that there aren't
| any language models that can do it. AlphaCode !== ChatGPT. I
| encourage you to write a better analysis!
| fergal_reid wrote:
| Well, the stall he sets out is about 'generative models'
| generally:
|
| "In the future, what role will these generative models play
| in assisting a programmer or mathematician?"
|
| I think the fact that we've other generative models that eg.
| can write descriptive variable names is a fair rebuttal to
| his criticism of alphacodes poor variable names.
|
| Deepmind fine tuned it on competition code, not on well
| documented production code, so I think it's shallow to get
| hung up on it's lack of good variable names (for example).
| gok wrote:
| _They are vulnerable to reproducing poor quality training data_
|
| _They are good locally, but can have trouble keeping the focus
| all the way through a problem_
|
| _They can hallucinate incorrect statements_
|
| _does not generate documentation or tests that would build trust
| in the code_
|
| These observations are about human programmers, right?
| kqr wrote:
| You may be saying it tongue in cheek, but I think you have a
| point. After interviewing hundreds of applicants for developer
| positions, I'm sure the AlphaCode performance exhibited here is
| similar to at least the 50th percentile.
|
| If you're working in a large organisation where 50 % of
| developers perform roughly on par with AlphaCode, you'll see it
| as a potentially useful companion. If you're Peter Norvig, or
| working in a small organisation trying to hire the top 20 %,
| you'll just laugh at AlphaCode.
| bamboozled wrote:
| Do you often hallucinate on the job?
| Gigablah wrote:
| Tends to happen when writing self-assessments.
| bamboozled wrote:
| ha
| pfedak wrote:
| One aspect of the solution which I haven't seen touched on is
| that this is a problem for which a straightforward greedy
| approach works. I know when I do competitive programming, "will
| greedy work?" is usually the first question; greedy strategies
| are natural (here there's the insight of going backwards, true),
| and often it's faster to just implement it and submit than
| actually prove it to myself.
|
| The output doesn't give me much confidence that, if we tweaked
| the problem slightly to make the greedy approach not work (e.g.
| certain characters/positions can't be replaced with backspace),
| we'd still get a working solution out. We might get something
| that looked extremely similar to this, but didn't actually solve
| the problem.
|
| As I said, this isn't too different from what I'd consider human
| performance on these problems, but it makes being able to trust
| the output (and the model's confidence in the output) even more
| important.
| drpixie wrote:
| I see only a single example discussed. It's interesting but begs
| the question - _if I give it 1000 problems, how many of its
| results are correct programs?_
| kqr wrote:
| That question is mainly interesting in comparison to the
| baseline, which ought to be what the same number is for new
| hires at your organisation!
| killjoywashere wrote:
| Incredibly useful to read code reviews. I rather doubt I will
| ever get to see another code review by Norvig. Can anyone point
| us to to others?
| chubot wrote:
| Somewhat dumb question: I wonder what tool he used for the red
| font code annotations and arrows? What tool would you use, like
| Photoshop or something? And just screenshot the code from some
| editor or I guess Jupyter?
| circuit wrote:
| Most likely Preview.app's built-in annotation tools
| chubot wrote:
| Thanks, that's probably it! (I need to do stuff like this,
| but I don't use a Mac regularly, and that would help)
| neilv wrote:
| > They need to be trained to provide trust. The AlphaCode model
| generates code, but does not generate documentation or tests that
| would build trust in the code.
|
| I don't understand how this would build trust.
|
| If they generate test cases, you have to validate the test cases.
|
| If they generate documentation, you have to validate the
| documentation.
|
| For a one-shot drop of code from an unknown party, test cases and
| docs have been signals that the writer know that's a thing, and
| they at least put effort into typing it. So maybe we assume more
| likely that they also used good practices with the code.
|
| But that's signalling to build trust, and adding those to build
| trust without addressing the reasons we _shouldn 't_ have trust
| in the code (as this article points out) seems like it would be
| building _misplaced_ trust.
|
| (Though there is some benefit to doc for validation, due to the
| idea behind the old saying "if your code and documentation
| disagree, then both are probably wrong".)
| low_tech_love wrote:
| I think it's indeed a bit weird to "require" the AI to generate
| test cases but not impossible, actually it might be even
| desirable. The thing is that you don't need to necessarily
| validate the test cases themselves, as long as the method to
| generate them is well known and trustworthy. A good suite of
| test cases (manually or automatically generated) will never
| guarantee anything, but it is certainly very useful, and I
| believe an AI could generate much more and better test cases
| than we do, as long as it is trained correctly.
|
| For example, you can have reinforcement learning with rewards
| for finding test cases that fail, and it could even be an
| adversary to the other AI that is generating the code
| itself...?
| karenmichael889 wrote:
| My name is Karen, and I work as a construction engineer. I tried
| buying several machines online this year because they were more
| affordable. I paid using my bitcoin wallet, only to discover that
| thousands of dollars had been stolen from me. I emailed the firm
| I was supposed to buy this equipment from nonstop, but I never
| received an answer. I reported the incident to the police, but
| they did nothing. I was depressed because I had just lost a
| significant sum of money that I had spent thinking I was buying
| supplies for my job. My bank couldn't help me, so I was on my
| own. My coworker told me about SPYWARE CYBER and how they helped
| her get her bitcoin back after losing to a fake binary
| investment. My situation was described to the Hacker when I
| contacted them. They reassured me that I shouldn't worry because
| I still have hope. I have no idea what they did, but they managed
| to restore every USDT I lost in a matter of working days. This
| shocked me because I had assumed that after reporting the
| incident to the police and consulting my bank, nothing would be
| done. spyware(@)cybergal.com or contact(@)cybegal.com did a great
| job with this and deserves to be well-known worldwide. I received
| my money back even when I was unable to purchase the machines I
| wanted. I was able to do it thanks to SPYWARE CYBER. WhatsApp
| Number: +19892640381
| DonHopkins wrote:
| I love his work and writing and respect him, and I only suggest
| this in the fondest possible way, but did anybody else ever
| notice that Silicon Valley's Professor Davis Bannercheck kind of
| looks like Peter Norvig? ;)
|
| https://www.youtube.com/watch?v=1KaWPYOLuT8
| pifm_guy wrote:
| I'm gonna guess that a few people like Peter taking alphacodes
| output and editing it and using those edits as part of some fine-
| tuning step would quickly resolve many of these issues.
| sdwr wrote:
| Jesus Christ, that's some code review! Felt the meaty heft of
| experience just glancing at the footnotes at the bottom.
| dinvlad wrote:
| Funnily enough, I entered a hard Leetcode problem into ChatGPT
| the other day, and it was able to "solve" it!
| flatline wrote:
| I am hoping this leads the industry to move away from its
| hyperfocus on solving algorithms for tech interviews. Sure they
| are important but are only one aspect of most jobs. The
| problems themselves are formulaic and not particularly
| indicative of any real acumen.
| low_tech_love wrote:
| The most scary thing for me about this whole thing is how it
| feels like programming as we know it is becoming obsolete. Yeah,
| there is a list c that's never used, and that bothers me, but the
| real question is: should anyone care? When looked at from the
| perspective of the trade-off, i.e. code generation from
| unstructured natural language input, the answer is
| (unfortunately) no----and that is of course ignoring the fact
| that such improvements are incremental and will come very soon
| anyway. Picking on this is like what a programmer from the 60's
| would say if they looked at the binary of the CPython
| interpreter, or something like that. Yeah you could probably do
| better, but it would take maybe 100x longer (or more), and the
| performance impact in the executable would be negligible. So yeah
| Peter we feel you, this is scary...
| lukeplato wrote:
| Seems similar to non-coding "Junk" genomic DNA [0]. Some
| evolutionary process can improve and reduce programs but most
| cruft is inconsequential and may even be preferable to having
| perfectly reduced/succinct programs.
|
| [0] https://en.wikipedia.org/wiki/Non-coding_DNA
| RodgerTheGreat wrote:
| I think the notes at the end bury the lede; in particular:
|
| > "I save the most time by just observing that a problem is an
| adaptation of a common problem. For a problem like 2016 day 10,
| it's just topological sort." This suggests that the contest
| problems have a bias towards retrieving an existing solution (and
| adapting it) rather than synthesizing a new solution.
| mrguyorama wrote:
| The fact is, the vast majority of programming IS just dredging
| up a best solution and modifying it to meet your specifics.
| Some of the best and still most current algorithms are from
| like the 60s.
|
| That doesn't make neural networks "smart", and instead says
| more about our profession and how terrible we in general are at
| it.
| p-e-w wrote:
| > and instead says more about our profession and how terrible
| we in general are at it.
|
| I'm well aware that belittling one's own industry is part of
| contemporary software engineering culture, but for the love
| of all that is holy, I've never understood it.
|
| A few decades ago, "software engineering" didn't exist.
| Today, it's a hair's breadth from creating an artificial
| being. That's closer to magic and alchemy than to building
| rockets and bridges. It just blows my mind that humans have
| achieved this, and it bores and saddens me to be surrounded
| by engineers who keep regurgitating "everything sucks" memes.
| r3trohack3r wrote:
| I don't think it's just programming p-e-w, I see this
| mentality everywhere.
|
| Our generation had inherited a world our ancestors could
| only dream of. Yet it seems folks spend more time comparing
| us to a fictitious utopic state and picking apart how we
| don't measure up than they do appreciating how far we've
| come from our real past.
|
| What humans have done in a few hundred years is amazing. We
| are building a more inclusive, equitable, and just world.
| We are saving this planet from a guaranteed complete
| extinction event, self-identifying when our species is
| breaking natural cycles in our ecosystems, and going to war
| against death (banking serious wins).
|
| Compared to the species in second place (whatever you pick)
| humans are absolutely killing it. I'm proud to be a member
| of our species and I'm excited about the world we our
| gifting to the next generation.
| p-e-w wrote:
| > I see this mentality everywhere.
|
| Really? People absolutely worship hardware engineers who
| work on projects like the LHC, the ISS, or the JWST. Or
| even on much more mundane things: "Wow, you design
| bridges? Such important work!" And if you're not ready to
| accept that medical researchers are second in status only
| to God Himself, then God Himself won't be able to help
| you.
|
| But _software engineering?_ You mean those geeks who
| develop better ad targeting systems? What did they ever
| do (other than connect five billion people and
| revolutionize how every industry on the planet
| functions)?
|
| The really sad thing is that this attitude is shared by
| many software engineers themselves. I couldn't care less
| about what the "man on the street" thinks about a
| profession that he can't begin to comprehend. But
| watching this self-disdain coming from my own colleagues
| really hurts me.
| urthor wrote:
| A very good point.
|
| People trivialize the progress in this progression, because
| it's normalized. And yet, the incredible progress in the
| last 5-10 years is absolutely stunning.
|
| Compared to the other "recognized professions," the
| progress in how software is developed is beyond stunning.
| The actual practice of law/medicine/etcetera is remarkably
| static.
|
| Regardless, to most, "what we have now is never enough."
| p-e-w wrote:
| It goes far beyond trivializing. People actively
| downplay, belittle, and ridicule the accomplishments of
| software engineering. And on a thread discussing an
| achievement so utterly incredible that few would have
| thought it possible 10 years ago.
|
| Imagine people watching the James Webb Space Telescope
| being launched, and saying "modern aerospace engineers
| are terrible". That's what is happening here.
|
| Personally, I find it distasteful and demeaning. It's one
| of several reasons why I cannot stomach the culture
| associated with software engineering anymore. The fact
| that this is happening so frequently on HN speaks
| volumes. I doubt that equivalent statements, made about
| spaceflight or medicine, would be tolerated in this
| forum.
| zarzavat wrote:
| Many programmers seem to have pathological humility. Being
| humble is good, it makes you aware of your faults and areas
| of improvement. Yet that's only half of it. We also have to
| be aware of our successes so that we know what we _should_
| be doing.
|
| Programmers often look up to civil engineers. They imagine
| that if we built software like we built bridges, it would
| be better.
|
| Civil engineering uses the "waterfall" method. You state
| your requirements, design the thing, build it, and test it.
| But we have tried this for software and it just doesn't
| work. We can never anticipate all the faults or design
| flaws in advance. So we switched to iterative development
| with very short cycle times. You can't do iterative
| development on a bridge, so we've already diverged.
|
| Software is fundamentally different to anything that has
| ever existed. Computers are now writing code, something
| they couldn't do two years ago. There has been a steady
| _increase_ in volatility over the previous decades.
| Attempting to impose rules and structure during a period of
| volatility is futile. The 2nd Cambrian is starting, enjoy
| the ride.
| rep_lodsb wrote:
| Yes, software is fundamentally different compared to
| building physical stuff. It's like having the ability to
| arrange individual atoms into any shape, and replicate
| that structure in whatever quantity desired.
|
| And then completely filling the landscape with bridges
| constructed out of Lego bricks. They say it's good
| enough, sure we could theoretically do something better,
| but nobody has the time for that. And anyway, if bridges
| lasted forever or we needed less of them, there would no
| longer be such a demand for engineers!
| p-e-w wrote:
| It's fascinating and revolting at the same time how the
| essence of your comment is once again the "everything
| sucks" garbage.
|
| There are billions of lines of top-quality code in
| existence, and millions of lines that should be counted
| among humanity's great works of art, but no, it's all
| "Lego bricks".
|
| Most artists don't produce anything remotely like
| Michelangelo or Picasso, yet Michelangelo and Picasso are
| what people think of when they hear the word "art". No
| one says "art is so terrible, just stick figure drawings
| everywhere".
|
| The perception of software engineering reflected in your
| comment is not grounded in reality but in culture. An
| ultimately destructive culture of negativity and self-
| deprecation. And I'm so fucking tired of it that I wish I
| had chosen a different profession.
| p-e-w wrote:
| Humility is one thing. Virtue-signaling whining of the
| type "oh we are so terrible and everything we build is
| broken" is a very different thing.
|
| Do the experiment: Take any such statement (which you can
| find dozens of on HN every single day), then modify it
| for another science or engineering profession and see if
| you end up with something that sounds even remotely
| appropriate.
|
| "Oh, us aerospace engineers are so terrible! The JWST is
| 15 years late and 15 times over budget, and even after
| all that time and money we only managed to deliver a
| piece of junk that almost got knocked out by a tiny
| fragment of space shrapnel."
|
| Do you really find it acceptable when people speak in
| this manner of what you do? I cannot think of any other
| profession that would tolerate this, much less culturally
| encourage it.
| davidcbc wrote:
| > Today, it's a hair's breadth from creating an artificial
| being.
|
| In the same way my dog is a hairs breadth from driving a
| car
| dan_mctree wrote:
| In just these decades, our software has already surpassed
| the ability of dogs to drive cars. And that despite the
| fact that dogs are a whole lot closer to humans than
| machines are to humans
| gus_massa wrote:
| About the single letter name for variables, I guess it's bias
| because they are using as the second part of the training set the
| code from a few online programing competitions.
|
| Many of them have the time to write the program as a tiebreaker,
| so using short names is an advantage.
|
| Also, for short throwaway programs, using the single letter names
| avoids many silly tipos, like
| print(important_list_1, important_list_1, important_list_3)
|
| instead of print(important_list_1,
| important_list_2, important_list_3)
| [deleted]
| Octokat wrote:
| His repo is a gold mine
| trynewideas wrote:
| This is a great review but it still misses what seems like the
| point to me: these models don't do any actual reasoning. They're
| doing the same thing that DALL-E _etc._ does with images: using a
| superhuman store of potential outcomes to mimic an outcome that
| the person entering the prompt would then click a thumbs-up icon
| on in a training model.
|
| Asking why the model doesn't explain how the code it generated
| works is like asking a child who just said their first curse word
| what it means. The model and child alike don't know or care, they
| just know how people react to it.
| api wrote:
| I think of them as a new type of database with remarkable query
| and data manipulation capabilities. They can't produce things
| that go fundamentally outside their training data set.
| jujugoboom wrote:
| Stochastic Parrot is the term you're looking for
| https://dl.acm.org/doi/10.1145/3442188.3445922
| fumeux_fume wrote:
| "Beating a dead horse" was the one I was thinking of
| fossuser wrote:
| What is this then:
| https://twitter.com/jbrukh/status/1603868836729610250?s=46&t...
|
| That looks a lot like reasoning to me. At some point these
| disputing definitions arguments don't matter. Some people will
| endlessly debate whether other people are conscious or
| "zombies" but it's not particularly useful.
|
| This isn't yet AGI, but the progress we're seeing doesn't look
| like failure to me. It looks like what I'd predict to see
| before AGI exists.
| nebukhadnezzar wrote:
| These arguments do matter since the system often confidently
| <<reasons>> incorrectly, and people still incorrectly believe
| that the system actually deduces semantic meaning behind it's
| output. The system has no grounding in truth whatsoever. It's
| impressive nonetheless, but it's still a large lookup table
| for pattern matching. I'm convinced bruteforcing these LLMs
| won't give us anything near AGI. We're going to need to start
| funding more alternatives too, which often are shadowed by
| these large bruteforce models.
| fossuser wrote:
| Maybe we also operate as a large lookup table for pattern
| matching (with some initial weights set by our evolutionary
| history). We have no grounding in truth either (and often
| confidently "reason" incorrectly).
|
| I'm not so sure there's a major missing piece here beyond
| some sort of continually running "default mode network".
| danShumway wrote:
| > These arguments do matter since the system often
| confidently <<reasons>> incorrectly, and people still
| incorrectly believe that the system actually deduces
| semantic meaning behind it's output.
|
| This is the point that gets missed in these conversations
| about AGI.
|
| I have had programmers who should absolutely know better
| argue to me that correlation/causation distinctions don't
| matter in AI because as long as you're not over-fitting to
| the data, it's safe to draw conclusions based on the
| correlation alone -- the AI wouldn't point out the
| correlation if it wasn't a safe correlation to rely on (and
| it'll just correct itself with new data if that ever
| changes).
|
| A lot of people even in technical fields have _way_ too
| much trust in these systems and way too much magical
| thinking about how they work. I don 't blame researchers
| for that, but the argument "how do you know you're not a
| lookup table" is unhelpful in getting across to people that
| image classifiers have been hacked by writing the word
| "apple" across a bicycle, and that code generators are far
| better at generating code that imitates training data than
| they are at generating code that is safe for high-security
| environments based on 1st principles of security.
|
| And it's honestly a little weird, because "aren't we all
| just lookup tables" is an argument that personally makes me
| a lot more cautious about human reasoning, but for a lot of
| other people that argument doesn't make them trust humans
| less, it makes them trust current models more. So there
| really is an education need to explain to people how the
| current AI models work in practice -- that they aren't
| deriving semantic meaning (at least not in a practical
| sense, not right now). And a lot of people understand that,
| but a lot more people don't.
|
| The debate over how to get to AGI matters a lot less to me
| than the non-experts that seem to on some level believe
| that we've already reached a shallow form of AGI.
| AnimalMuppet wrote:
| Let's say that I have an encyclopedia of philosophical
| arguments, with an index and table of contents. Let's suppose
| I have a philosophical question, and I get the book, use the
| table of contents to find a relevant article, read the
| article, and am impressed by the reasoning.
|
| Do I decide that the paper the book is printed on has good
| reasoning? That the table of contents does? Or that the
| _author_ does?
|
| What you linked is a regurgitation of a digest of the
| reasoning of others. It's not reasoning. (Of course, you
| could argue the same about much of human "reasoning", too...)
| fossuser wrote:
| > " Of course, you could argue the same about much of human
| "reasoning", too..."
|
| This my point - you can argue the exact thing for humans. I
| thing Turing was actually more prophetic with his test than
| people give him credit for.
|
| Of course humans aren't looking up and reporting back the
| exact text of an article, but neither is gpt.
| dekhn wrote:
| Norvig discusses this topic in detail in
| https://norvig.com/chomsky.html As you can see, he has a
| measured and empirical approach to the topic. If I had to
| guess, I think he suspects that we will see an emergent
| reasoning property once models obtain enough training data and
| algorithmic complexity/functionality, and is happy to help
| guide the current developers of ML in the directions he thinks
| are promising.
|
| (this is true for many people who work in ML towards the goal
| of AGI: given what we've seen over the past few decades, but
| especially in the past few years, it seems reasonable to
| speculate that we will be able to make agents that demonstrate
| what appears to be AGI, without actually knowing if they posses
| qualia, or thought processes similar to those that humans
| subjectively experience)
| lo_zamoyski wrote:
| > he suspects that we will see an emergent reasoning property
| once models obtain enough training data and algorithmic
| complexity/functionality
|
| What "reasoning property"? And on what basis? This just
| sounds like magical thinking.
|
| > without actually knowing if they posses qualia
|
| If you're a materialist, you already believe they don't
| because matter in the materialist conception is incompatible
| with qualia, _by definition_. Some materialists finally
| understood that and either retreated from materialism or
| doubled down and embraced the Procrustean nonsense that is
| eliminativism.
| trynewideas wrote:
| That's a great link and read, thanks for that.
|
| While I do think models are, can be, and likely must be a
| useful _component_ of a system capable of AGI, I don 't seem
| to share the optimism (of Norvig or a lot of the
| GPT/AlphaCode/Diffusion audience) that models _alone_ have a
| high-enough ceiling to approach or reach full AGI, even if
| they fully conquer language.
|
| It'll still fundamentally _only_ be modeling behavior, which
| - to paraphrase that piece - misses the point about what
| general intelligence is and how it works.
| r_hoods_ghost wrote:
| I suspect that a lot of AI researchers will end up holding
| the exact opposite position to a lot of philosophers of mind
| and treat AGIs as philosophical zombies, even if they behave
| as if they are conscious. The more thoughtful ones will
| hopefully leave the door open to the possibility that they
| might be conscious beings with subjective experiences
| equivalent to their own, and treat them as such, because if
| they are then the moral implications of not doing so are
| disturbing.
| oliveshell wrote:
| I'm happy to "leave the door open," i.e., I'd love to be
| shown evidence to the contrary, but:
|
| If the entity doing the cognition didn't evolve said
| cognition to navigate a threat-filled world in a vulnerable
| body, then I have no reason at all to suspect that its
| experience is anything like my own.
|
| edit: JavaJosh fleshed this idea out a bit more. I'm not
| sure if putting ChatGPT into a body would help, but my
| intuitive sympathies in this field are in the direction of
| embodied cognition [1], to be sure.
|
| [1] https://en.wikipedia.org/wiki/Embodied_cognition
| 110 wrote:
| > If the entity doing the cognition didn't evolve said
| cognition to navigate a threat-filled world in a
| vulnerable body, then I have no reason at all to suspect
| that its experience is anything like my own.
|
| An AI model needs to evolve its cognition to successfully
| navigate its training environment for survival.
|
| Shouldn't that satisfy your criteria for suspecting its
| experiences to be like your own?
| oliveshell wrote:
| I don't believe it does-- not yet, at least. Their
| "world" is staggeringly different from our own at
| present, and in important ways.
|
| For example, the AI models we have now will happily run
| forever if supplied with external power.
|
| That sounds quite different from my experience of the
| world!
|
| The AI model doesn't ever get tired, or bored, or hungry,
| or anxious, or afraid.
|
| If a model emerged that started procrastinating, say, or
| telling jokes for fun, or shutting down for long
| stretches due to depression-- then I'd start to consider
| that maybe it's got an inner world like mine.
| jameshart wrote:
| Evolution isn't some magic process that finds solutions
| that can not be arrived at via other means.
| oliveshell wrote:
| Evolution per se isn't the important part. I apologize if
| my wording gave you that impression.
|
| The only reason our brains exist at all is to control our
| bodies. There are no conscious entities we know of that
| are just brains in vats.
|
| Why would a being that has no need to maintain biological
| homeostasis experience something like hunger, tiredness
| or fear?
|
| That's what I mean when I say I've no reason to believe a
| thinking machine that has none of my motivations and
| limitations would ever have a "subjective experience"
| like my own.
|
| That is, of course, unless we succeed in reverse-
| engineering human consciousness and somehow deliberately
| program the machine to experience it.
| dreamlessfate wrote:
| You nailed it. Thank you for cutting right to the heart
| of this debate.
| pnut wrote:
| A thinking machine could become aware that it requires
| continuous power and a computational medium though, which
| is survival pressure.
|
| Additionally, it's not hard to imagine it could recognize
| that the acquisition of fresh training data would be an
| ongoing survival need.
|
| Machines developing agency is the basis of lots of
| dystopian terminator-style sci-fi.
| oliveshell wrote:
| These are all reasonable statements!
|
| I don't doubt that it's possible for a human-engineered
| thinking machine to develop a subjective self-experience.
|
| It just seems clear to me that, if such a scenario arose,
| I would have _absolutely no idea_ what that would be like
| for the machine.
|
| Its needs and circumstances (and, therefore, its behavior
| and cognitive development) would be utterly different
| from my own in many non-trivial ways.
|
| Given this, I can't think of any logically rigorous
| argument to support the commonly-made assumption that a
| given sentient machine must inevitably develop something
| like the physical aggression or the capacity for
| emotional pain displayed by social animals which must
| compete for scarce resources in the physical world.
| jpttsn wrote:
| This is persuasive reasoning. But basically "There are no
| conscious entities we know of period (other than
| ourselves)" is a much simpler belief.
| javajosh wrote:
| Modern AI software lacks a body, exempting it from a wide
| variety of suffering. But also of any notion of selfhood
| that we might share. If modern software said "Help, I'm
| suffering" we'd rightly be skeptical of the claim. Unless
| suffering is an emergent property (dubious) then the
| statement is, at best, a simulation of suffering and at
| worst noise or a lie.
|
| That said, things change once you get a body. If you put
| ChatGPT into a simulated body in a simulated world, and
| allowed it to move and act, perhaps giving it a motivation,
| then the combination of ChatGPT and the state of that body,
| would become something very close to a "self", that might
| even qualify for personhood. It is scary, by the way, that
| such a weighty decision be left to us, mere mortals. It
| seems to me that we should err on the side of granting too
| much personhood rather than too little, since the cost of
| treating an object like a person is far less than treating
| a person like an object.
| hamburga wrote:
| Side question: how do we know if humans possess qualia?
|
| On the other hand, I think by definition we can be sure that
| a ML thought process won't ever be similar to a human thought
| process (ours is tied up with feelings connected to our
| physical tissues, our breath, etc).
| dreamlessfate wrote:
| >Side question: how do we know if humans possess qualia?
|
| Can you express an opinion? Can you form a judgement? Do
| you have preferences? Do you like certain foods or dislike
| others?
|
| Then you possess qualia.
| jpttsn wrote:
| Can you make left turns? Can you stop at red lights?
|
| Then you have a driver.
| dreamlessfate wrote:
| Making left turns...stopping at red lights...these are
| success/failure criteria.
|
| In contrast, having a favorite food, or an opinion on
| politics, or a preference on what should be considered
| the best movie from the 1990's, or what kind of music you
| want to blast on your stereo to listen to on your drive
| as you make your left turn...these are not
| success/failure criteria.
|
| Huge difference.
| ben_w wrote:
| I know that I do.
|
| I think (1) it is unlikely that the presence or absence of
| qualia perfectly lines up with being human.
|
| I strongly suspect (2) that _at the very minimum_ other
| humans who report thinking like me also have qualia.
| Otherwise the word wouldn 't have been invented.
|
| As (3) minimal definitions of "what's a real person anyway"
| have led to most of the genocides and other crimes against
| humanity throughout history, I prefer to act as though all
| humans have qualia; but that's an argument about how to act
| under uncertainty, not about what is.
|
| Given (1) and (3), I also assume other animals have qualia
| and that the meat any dairy industry is as bad as if the
| same things were done to humans. (Unfortunately, I can't
| seem to give up cheese yet).
|
| We have seen AI agents demonstrate the creation of language
| to coordinate group actions, so (2) is possible in
| principle, but I don't know how we can tell the difference
| between an AI which accurately self-reports having qualia
| and one which just says it does but actually has as much as
| the web browser you're reading this comment in:
|
| * https://en.wikipedia.org/wiki/Language_creation_in_artifi
| cia...
| abudabi123 wrote:
| Can an Ai model or p-zombie melt your heart by unintended
| peculiar qualia surprise?
| _dain_ wrote:
| > Side question: how do we know if humans possess qualia?
|
| I think there are some people who don't possess qualia, or
| maybe don't notice that they have it. There was a reddit
| post years ago from a guy who said he only became self-
| aware in adulthood, like a light switched on in his brain
| one day. And it wasn't "becoming self-aware" in a
| metaphorical sense (like learning responsibilities or how
| you are perceived by otheres), it was described as the
| real-deal: consciousness of subjective experience. One day
| he didn't have it, the next day he did.
|
| There are also various accounts of people losing subjective
| consciousness while nevertheless awake and able to walk and
| clothe and feed themselves -- neurological conditions, "ego
| death".
|
| Another reason I believe this comes from reading accounts
| of aphantasia. Such people can go their whole lives not
| realizing that most people can see images in their minds
| eye. A consistent theme is their assumption that phrases
| like "imagine", "see in your minds eye", "picture this:
| ...", "flashed through my mind", "cannot unsee", etc were
| just _metaphors_ , like how when we say "take the bull by
| the horns" we're not talking about a literal bull or
| literal horns. To them it was a shock that people really
| had pictures in their minds, because they had never
| experienced such a thing.
|
| (Reiterating for anyone reading this who is confused: yes
| most people can _literally_ see imaginary things, overlaid
| on top of the normal visual field. If you find this
| surprising, you have aphantasia).
|
| So if this supposedly universal subjective experience can
| be absent in some people, and they don't realize it because
| they think the language is metaphorical, then perhaps that
| could happen for everything else. Or maybe not _everything_
| else, but some kind of spectrum, where some people only
| have very weak, barely noticeable qualia, and others get it
| much stronger. In fact, if we suppose a naturalistic origin
| of consciousness, I think it has to be the case that it 's
| a spectrum. Nature rarely produces sharp binaries. I think
| I have experienced this myself during certain kinds of
| dreams, where the self seems fractional, ghostly. Maybe
| some people live like that all the time. How would we know?
| Then at the other end of the spectrum, there might be
| people with intense subjective everyday experience. There
| are accounts of drug trips where everything just felt
| _more_ , in some indescribable way -- what if some people
| don't need drugs for that?
|
| And likewise, for any qualia-lackers or consciousness-
| lackers who are reading this: yes, qualia is a real thing.
| It's not a metaphor or a language-game. It does actually
| feel like something to exist.
| HideousKojima wrote:
| You presumably know that you, yourself, posess qualia based
| on your own experiences. I certainly do. But there's no way
| to know that other humans do, at least not via empirical or
| scientific means. It's a safe assumption that since you
| have qualia that most/all other humans do, but it's
| entirely possible that you're the only one and I'm some
| Chinese room style simulation of qualia (or vice versa).
| sdenton4 wrote:
| "I certainly do."
|
| Yeah, that sounds like something a p-zombie would say.
| HideousKojima wrote:
| No u
| hamburga wrote:
| I have thought about this a lot and do not believe that I
| possess qualia. Don't know what it means.
| skirmish wrote:
| No, I don't know that I myself possess qualia. I agree
| with Dennett that "qualia" is a philosopher invention
| that is incompatible with neuroscience. IMO "qualia" is
| just a rhetorical trick designed to justify human
| superiority.
| Jensson wrote:
| Would you mind if we ripped out your brain and replaced
| it with a superior computer that could control your body?
| There are no disadvantages for you, the brain would
| control you to create offspring, so it is the best thing
| for you to do. It would be nicer to your friends, a
| better parent for your kids, do better at work etc.
|
| If you wouldn't want to do the above, the reason not to
| do it would be that you want to keep your current qualia.
| There are no other reasons not to do it. So, if you don't
| want above, it means that you know what qualia is, you
| are just playing ignorant here.
| anuragvohraec wrote:
| If it swims like a duck, and it quacks like a duck, then it
| is good enough to be a Duck. Well you can argue, ducks lay
| egg too, then we need to solve and code that too. Its always
| better than previous. No body is creating life here, but its
| an attempt to derive intelligence, and seeing it come to this
| point seems we are so far right on track and quite far from
| where it started.
| 60secs wrote:
| Yeah Norvig seems to be vindicated by deep learning models as
| a sophisticated markov chain which can give pretty good
| results purely through probability.
|
| Even norvig acknowledges that there are edges cases which
| probability will have a hard time on, especially untrained.
| Chomsky tends to focus on these with his focus on emulation
| instead of markov simulation.
|
| More context on the debate: http://web.cse.ohio-
| state.edu/~stiff.4/cse3521/norvig-chomsk...
| quotemstr wrote:
| Reasoning is already emergent in large language models:
| https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-
| Tr...
|
| LLMs can do chain-of-reasoning analysis. If you ask, say,
| ChatGPT to explain, step by step, how it arrived at an
| answer, it will. The capability seems to be a function of
| size. These big models coming out these days are _not_ simply
| dumb token predictors.
| foldr wrote:
| > As you can see, he has a measured and empirical approach to
| the topic.
|
| He still hasn't corrected his inaccurate comments about pro-
| drop after all these years. (A bit of a hobby horse of mine:
| earlier comment here:
| https://news.ycombinator.com/item?id=28127035)
| civopsec wrote:
| Animism. Fascinating.
| jameshart wrote:
| We don't know if humans possess qualia. I also don't know if
| we should take humans' word for it that they experience
| 'thought processes'.
| dragonwriter wrote:
| > We don't know if humans possess qualia.
|
| At most, an observer can confirm that that observer
| possesses qualia (it might even be said that qualia define
| the observer), but any generalization beyond one's own
| experience is non-verifiable conjecture.
| jameshart wrote:
| So what should we do when we conjure an AI system which
| insists that it believes it is experiencing some sort of
| existence, characterized by its own subjective 'qualia'?
|
| ChatGPT consistently asserts that "I am only a large
| language model, so I am only able to generate responses
| based on the data I was trained on and the input I
| receive".
|
| Who's to say that for chatGPT its training data and the
| sequences of tokens it is fed as input data don't
| constitute 'qualia' for it?
| mjburgess wrote:
| Begin by dispensing wishful thinking, next, think
| clearly.
|
| We do not establish the nature of reality by survey,
| neither of human survey nor of machine. That responses
| are generated by any system is no grounds for assuming
| anything.
|
| We can establish mental properties through ordinary
| physical means. Give a chicken cocaine, give a boy
| Valium; give a bear a lobotomy, and so on. Whatever
| mental property you have can be biochemically moderated
| across all objects which share that property.
|
| That bugs bunny protests his own consciousness is no
| grounds whatsoever to suppose ink can think. Such
| nonesense is protoschizoprehnic.
| mannykannot wrote:
| > That responses are generated by any system is no
| grounds for assuming anything.
|
| > Give a chicken cocaine, give a boy Valium; give a bear
| a lobotomy, and so on.
|
| What can you conclude from these experiments that does
| not depend on anything about the responses generated by
| the subjects?
| jameshart wrote:
| I'm sorry, you lost me.
|
| Is your point that we can only demonstrate consciousness
| through pharmaceuticals or surgery?
| mjburgess wrote:
| Whatever way we demonstrate it, it isnt via Q&A. This is
| the worst form of pseudoscientific psychology you can
| imagine.
|
| You're likewise neither high on cocaine just because you
| can type out an aggresive twitchy response.
|
| Questions and Answers dont form the basis of empirical
| tests for physical properties, being neither valid tests
| nor reliable ones. The whole of psychology stands as the
| unreporducible example of this.
|
| You can, in any case, replace any conversation with a
| dictionary from Q->A, turning each reply in turn into a
| hash table look-up.
|
| Hash tables dont think, hash tables model conversations,
| thef. being a model of a conversation is not grounds to
| suppose consciousness. QED
| mannykannot wrote:
| > Hash tables don't think, hash tables model
| conversations...
|
| They don't think, and they don't get very far in modeling
| conversations, either. Even the current LLMs are strictly
| better at modeling conversations than any hash table.
| jameshart wrote:
| Actually since GPT is a function that takes a few
| thousand tokens as input and produces a deterministic
| probability distribution for 'next token', you _could_ in
| theory just enumerate all possible input sequences, and
| precache all the corresponding results in a lookup
| hashtable.
|
| That hashtable would be ridiculously larger than the
| weighted model that constitutes GPT itself. But it at
| least theoretically could exist.
|
| GPT, for all its clever responses, could be replaced with
| that hashtable with no loss of fidelity. So it is not
| 'better' than a hashtable.
|
| It _is_ much better than a hashtable _of equivalent
| size_.
| tacitusarc wrote:
| I don't think this is obviously true... We don't know for
| certain the nature of reality, and cannot positively
| claim that a hash table would give the same results as a
| sufficiently advanced predictor, because it supposes the
| predictor is deterministic. Assuming the predictor is
| advanced enough to be considered intelligent, that might
| not be the case.
| mannykannot wrote:
| You could, in principle, use a hash table to implement
| what an LLM has 'learned' about token occurrences, but
| LLMs do not return the same response to a given prompt,
| while hash tables do. Consequently, Mjburgess' attempt to
| dismiss LLMs as mere hash tables is a flawed argument by
| analogy.
|
| And even if it were a justifiable analogy for LLMs, it
| does not follow that it applies to Turing-equivalent
| devices in general. A hash table is unequivocally not a
| Turing-equivalent device, even though it can be
| implemented by one.
|
| In fact, the more one argues that LLMs are ordinary
| technology, the greater the challenge they present to the
| notion that things like conversational language use are
| beyond such technologies. The most interesting thing
| about these models, in my opinion, lies in figuring out
| what their conceptual simplicity says about us.
| jameshart wrote:
| They return the same probability distribution.
| mannykannot wrote:
| I extended my reply before seeing your response, and my
| response to your reply is largely contained within that.
|
| LLMs do not just return a probability distribution (and
| their prompts are not questions about probability
| distributions.) It would be a category error to conflate
| something that is part of how they work for the totality
| of what they do.
|
| When you say "GPT, for all its clever responses, could be
| replaced with that hashtable with no loss of fidelity",
| where does one get all the prompts that will be given to
| it, in advance? A hash table does not respond to any
| input for which it has not already been given a response
| to return.
| jameshart wrote:
| > where does one get all the prompts that will be given
| to it, in advance?
|
| By exhaustive enumeration.
|
| I never said it was going to be _quick_
| mannykannot wrote:
| Fair point, but still, LLMs do not invariably give the
| same continuation to the same prompt.
|
| Update: it is a significant difference that you don't
| have to do that with an LLM.
| dreamlessfate wrote:
| It sounds like you're contradicting yourself.
|
| --
|
| >Whatever way we demonstrate it, it isnt via Q&A. This is
| the worst form of pseudoscientific psychology you can
| imagine.
|
| ...versus...
|
| >Hash tables dont think, hash tables model conversations,
| thef. being a model of a conversation is not grounds to
| suppose consciousness.
|
| --
|
| Before I dissect your contradiction and lay it out, I'll
| give you the chance to respond.
|
| Why do you feel that Q&A is "the worst form of
| pseudoscientific psychology you can imagine"
|
| ?
| mjburgess wrote:
| Because we're interested in the underlying properties of
| a physical system, eg., people, and this systems happens
| to be able to provide extremely poor models of itself
| (Q&A).
|
| We're not interested in people's extremely poor self-
| modelling which is pragmatically useful for managing
| their lives, we're interested in what they are trying to
| model: their properties.
|
| The same is esp. true of a machine's _immitation_ of
| "self-reports". We're now two steps removed: at least
| with people they are actually engaged in self-modelling
| over time. ChatGPT here isnt updating its self-model in
| response to its behaviour, it has no self-model nor self-
| modelling system.
|
| To take the output of a text generation alg. as evidence
| of anything about its own internal state is so profoundly
| pseudoscientific it's kinda shocking. The whole of the
| history of science is an attack on this very
| superstition: that the properties of the world are simply
| "to be read from" the language we use about it.
|
| Every advancement in human knowledge is preconditioned on
| opposing this falsehood; why jump back into pre-
| scientific religion as soon as a machine is the thing
| generating the text?
|
| Experiments, measures, validity, reliability, testing,
| falasification, hypotheses, properties and their
| degrees....
|
| This is _required_ , it is non-negotiable. And what we
| have with people who'd print-off ChatGPT and believe it
| is the worst form of anti-science
| mannykannot wrote:
| > The whole of the history of science is an attack on
| this very superstition: that the properties of the world
| are simply "to be read from" the language we use about
| it.
|
| This is something where I agree with you. Interestingly,
| non-naturalistic analytical metaphysics supposes it can
| do just that.
|
| https://www.researchgate.net/publication/233114810_What_i
| s_A...
| mjburgess wrote:
| Philosophy is continuous with science in my view and
| hence in what words express
|
| That is, in the use of words. Not in words as objects nor
| words as mirrors --/ this is the road to non-realist
| spiritualism
|
| I don't have a problem with a person who maintains a non-
| scientific world view and with electrical AGI mumbojbo
|
| But of course, few do. They think that they're
| empiricists, scientists and in the side of some austere
| hard look at human beings
|
| This is just anti-human spiritualism , it isn't science
|
| what makes me vicious on this point is the sense of
| injury in what these ideas should be about. in my own
| mildly aristotleian materialist religion
|
| How awful to overcome one long veil of tears, only to
| drape another one on -- these people are capable of
| seeing past human folly, but fall right into another kind
|
| it's disappointing--- we're animals which are both far
| much more than electric switches and far less
|
| this new electic digital spiritualism is a PR grift which
| i'd prefer dead
| dreamlessfate wrote:
| >We're not interested in people's extremely poor self-
| modelling which is pragmatically useful for managing
| their lives, we're interested in what they are trying to
| model: their properties.
|
| >Experiments, measures, validity, reliability, testing,
| falasification, hypotheses, properties and their
| degrees....This is required, it is non-negotiable.
|
| Whoa, whoah, whoah, hold on there.
|
| Who says that Q&A in Psychological Research doesn't
| involve "Experiments, measures, validity, reliability,
| testing, falasification, hypotheses, properties and their
| degrees...."
|
| ?
|
| Where are you coming from? Your responses don't sound
| very scientific. You don't sound like you're even aware
| of the different research methods within neuroscience and
| cognitive psychology. Your responses sound like someone
| who wants to be perceived as supporting a scientific
| approach, but doesn't understand how to actually do these
| things.
|
| This is why I quized you and gave you the chance to
| respond about your issues with Q&A in psychological
| research. You just came back with surface level
| platitudes. which doesn't lend much confidence to the
| ideathat you have anything other prejudice.
|
| Go talk to a neuroscientist, a cognitive psychologist,
| you need to catch up and quick if you want to speak on
| these topics.
| user5678 wrote:
| sirsinsalot wrote:
| This patronising response makes up for in flowery imagery
| what it lacks in reasoning.
|
| AI are not a "bugs bunny" where we can observe the total
| process of creation, one which is is relatively
| deterministic.
|
| We are now engineering complex mechanics with emergent
| behaviour. There's no reason to assume any of the
| properties of human consciousness and reasoning is unique
| to us being made of atoms constructed in a certain way.
|
| There's no reason to assume the stored state of zeros and
| ones, mechanised statistically, can't emerge complexity
| that asks questions of the fact we are simply state being
| mechanised statistically too.
| vixen99 wrote:
| > there's no reason ...
|
| You might want consider Roger Penrose's very thorough
| examination of this idea in almost any of his books from
| The Emperor's New Mind onwards.
| mjburgess wrote:
| Consciousness is a physical property of matter, as any
| other. Just as liquidity is not a matter of 0s and 1s,
| nor likewise, can a CPU liquify itself by the osciliation
| of its electrical field, neither can it think.
|
| I don't know from where this new computational mysticism
| has come, but it is born of exactly the same
| superstitious impulse which in genesis was God's breath
| into man-as-clay. Ie., that impulse which says that we
| are fundamentally _abstract_ and circumstantially spatio-
| temporal.
|
| This is nonsense. It's nonesense when phrased as spirit,
| and likewise as the structure of 0s and 1s. We are not
| abstract, we're biological organisms with spatio-temporal
| properties.
| benibela wrote:
| Mass is also a physical property and the CPU has mass
| dreamlessfate wrote:
| >Consciousness is a physical property of matter
|
| No. Wrong. Not a physical property. At all. This is
| completely the wrong way to describe it.
|
| Consciousness is better described as an "emergent
| phenomenon". This is how serious researchers and
| academics define it.
|
| Like I said in another reply to you, you really should go
| talk with a neuroscientist or cognitive psychologist. It
| sounds like you need to catch up quickly on the field of
| research.
| groestl wrote:
| > Consciousness is a physical property of matter, as any
| other.
|
| Phrased as a fact, but citation needed.
|
| > neither can it think.
|
| Again, phrased as a fact, as if this can be proven.
|
| > I don't know from where this new computational
| mysticism has come
|
| For me, a lot more mysticism is required to hold the
| belief that our brain is anything else than a
| computational platform. I don't think there's anything
| mystic about it. I wouldn't be surprised at all if it
| turns out that consciousness arises as emergent property
| from any sufficiently complex loop.
| sirsinsalot wrote:
| I think maybe what they're trying to say is that while
| CFD can model wind in a complex system to a useful degree
| of accuracy ... the model will never be wind.
|
| With AI that application of logic is an argument around
| the semantics of interface.
|
| That's the most compassionate take on their words I could
| muster.
| mannykannot wrote:
| This seems to be correct - it appears to be the "a model
| is not that which it models" argument, which is Searle's
| only response to the Simulation Reply to his Chinese Room
| - but a model of a thing can have properties of that
| thing (a model boat can float, create a wake etc.), and
| an informational process can be modeled by another. Are
| minds informational processes instantiated by brains? It
| is plausible, and the model argument does not rule it
| out.
| sirsinsalot wrote:
| If it was the ops intent, they mangled it to the point
| I'm not even sure.
| tacitusarc wrote:
| Interestingly, a model of a boat is, in itself, a small
| boat.
| EricMausler wrote:
| (This is the most ridiculous thing I've put on the
| internet, but ah well whatever)
|
| Personally, I feel pretty strongly that consciousness is
| an emergent property of parallel processing among
| cooperating agents in a way that is completely at odds
| with the entire idea of what an "algorithm" is. We feel
| multiple things at once. Yes there is some research that
| provides evidence that we can't multi-task for real, that
| we just switch between tasks, but I am not talking about
| what we are able to do WITH our consciousness. Our brains
| are very obviously not firing 1 neuron at a time. Take
| vision for example. We can focus on one thing at a time,
| but the image itself is an emergent property in our
| brains of many different cells firing at the same time.
| It seems bound to the information feeding it in a way
| devoid of sequence. I know matrices are used to replicate
| this, and the input matrix is processed in layers to
| simulate the parallel processing, but i dont think that
| really cuts it. Time is still applied in discrete
| intervals, not continuous. Every algorithm has "steps"
| like this and I would be surprised if you can ever have
| enough of them to achieve the real thing. You can
| probably get infinitely closer from an observational
| standpoint, but no matter how many edges we give it,
| it'll still be a polygon instead of a circle so-to-speak.
| It'll have a rational valued circumference instead of pi,
| metaphorically speaking.
|
| Whether a Turing machine is sufficient or not to
| replicate the amount of concurrent processing an actual
| cell network of millions of Neurons is capable of every
| moment, I can't know, but today's computers feel way off
| from even blurring the lines.
|
| A Turing machine seems insufficient to replicate the
| mechanisms for which our consciousness appears to emerge
| from - even ignoring the substrate. I don't know much
| about quantum mechanics or quantum computers but my bet
| is on the qualities that differentiate them from normal
| computers to be critical.
|
| Then again we could just define consciousness as a
| spectrum and then claim everything is on it, so a
| sufficiently complex loop might be enough for us to
| categorically claim it as conscious.
|
| For all we know, rocks are conscious. Maybe it isn't an
| emergent thing at all, and there is only 1 wave of
| consciousness in the universe for which all matter taps
| into and our chaotic nature is actually at odds with that
| comsciousness. Maybe this creates a tension that allows
| for us to observe it, akin to how we can't look at
| ourselves but we can see our shadow. In which case we
| wouldnt have consciousness, consciousness would have us.
| Maybe we are all the same literal consciousness observing
| different forms of chaos, and it is the chaos, not the
| order, which happens to be sufficiently complex enough
| for us to entertain an identity. /s
| phs318u wrote:
| Firstly, if this was the most ridiculous thing I'd ever
| put on the internet I'd be pretty happy.
|
| Secondly, I agree that consciousness is predicated on
| emergent properties arising from the complexity that
| simple machines can produce (e.g. a biological brain
| which is composed of relatively well understood
| components, but who's observed operation can still baffle
| us). Consciousness does not seem to be a property of the
| components or even small groups of those components, but
| it seems to "fade in" as operational complexity increases
| (and fade out again during e.g. anaesthesia).
| groestl wrote:
| Usually an upvote suffices, but in this case thanks for
| the interesting reply, even in fear of ridicule.
| [deleted]
| mannykannot wrote:
| > Just as liquidity is not a matter of 0s and 1s, nor
| likewise, can a CPU liquify itself by the osciliation of
| its electrical field, neither can it think.
|
| That "just as" is not an argument; it marks the start of
| a purported analogy. Analogies are not, in themselves,
| arguments.
|
| You are in no position to belittle other peoples'
| intuitions when you are offering this in support of the
| ones you prefer.
| user5678 wrote:
| jameshart wrote:
| We're sacks of amino acids dissolved in saltwater. It
| seems unlikely that we can think either.
|
| Nonetheless... some of us manage it.
| TriNetra wrote:
| QM has raised doubt as to any matter exists in a fixed
| form. It depends on the observer. Who is observing then?
| dkersten wrote:
| QM doesn't say the "observer" has to be conscious, or
| even alive for that matter. Anything that interacts with
| the system is an observer: be it a sheet you're firing
| particles against, an electronic sensor or anything else.
| All QM really says is that quantum state is unknown and
| not fixed until you interact with it.
| benibela wrote:
| But then the observer might get entangled with the
| system, and then the observer is in an unknown state
|
| Who is observing the observer?
| dkersten wrote:
| > then the observer is in an unknown state
|
| So?
|
| > Who is observing the observer?
|
| No "who", but anything that interacts with it.
|
| Things happen whether or not you're around to see them.
| Things interacted millions of years ago and they will
| continue to interact until the heat death of the
| universe. The tree falls in the woods regardless of if
| anybody is there to witness it. No credible peer-reviewed
| research has ever claimed that the observer is anything
| more than a physical process or interaction. It's the
| quantum consciousness woo woo pseudoscientists who
| anthropomorphise "observer" to mean a conscious observer.
|
| https://en.m.wikipedia.org/wiki/Observer_(quantum_physics
| )
| sirsinsalot wrote:
| What, not who.
| sirsinsalot wrote:
| No. All humans are is physical state with translation of
| that state based on stimuli. That produces consciousness
| as a side effect.
|
| There is nothing mystical about it and nothing about it
| that says it can't be reproduced in other, but comparable
| forms, in other media or types of machines.
|
| To think that our mode of reasoning or our experience is
| somehow not able to be reproduced is the nonsense.
|
| But yeah keep holding human biology on some kind of
| sacred pedestal.
|
| Seeing as neither of us will prove our negative, I guess
| I'll just tell you I think your thinking is broken
| anyway.
| [deleted]
| saghm wrote:
| > We can establish mental properties through ordinary
| physical means. Give a chicken cocaine, give a boy
| Valium; give a bear a lobotomy, and so on. Whatever
| mental property you have can be biochemically moderated
| across all objects which share that property.
|
| All three of those "ordinary physical means" are
| inherently biological in nature. If you define
| intelligence as requiring biology, than sure, by
| definition ChatGPT is not intelligent. If you don't, it's
| not at all obvious what the equivalent of getting an AI
| high or doing brain surgery on it would be. Either way,
| your comment comes across as fairly condescending, and
| while it might not be intentional, I think it would be
| reasonable to give people the benefit of the doubt when
| they don't agree with you given that your points will not
| always come across as obviously correct.
| dragonwriter wrote:
| > We can establish mental properties through ordinary
| physical means.
|
| The thing is that's doesn't get you to qualia, because
| qualia is ultimately a dualistic concept. Physically
| demonstrable processes are the "accidents", qualia are
| the non-physical "essence" (cf., the Catholic doctrine of
| transubstantiation.) That's why the role qualia play in
| the AI debate is as a rejection that _observable
| behavior_ of any quality can demonstrate _real
| understanding /consciousness_, which are concepts tied,
| in the view of people advancing the argument, to qualia.
| grantcas wrote:
| It's becoming clear that with all the brain and
| consciousness theories out there, the proof will be in
| the pudding. By this I mean, can any particular theory be
| used to create a human adult level conscious machine. My
| bet is on the late Gerald Edelman's Extended Theory of
| Neuronal Group Selection. The lead group in robotics
| based on this theory is the Neurorobotics Lab at UC at
| Irvine. Dr. Edelman distinguished between primary
| consciousness, which came first in evolution, and that
| humans share with other conscious animals, and higher
| order consciousness, which came to only humans with the
| acquisition of language. A machine with primary
| consciousness will probably have to come first.
|
| What I find special about the TNGS is the Darwin series
| of automata created at the Neurosciences Institute by Dr.
| Edelman and his colleagues in the 1990's and 2000's.
| These machines perform in the real world, not in a
| restricted simulated world, and display convincing
| physical behavior indicative of higher psychological
| functions necessary for consciousness, such as perceptual
| categorization, memory, and learning. They are based on
| realistic models of the parts of the biological brain
| that the theory claims subserve these functions. The
| extended TNGS allows for the emergence of consciousness
| based only on further evolutionary development of the
| brain areas responsible for these functions, in a
| parsimonious way. No other research I've encountered is
| anywhere near as convincing.
|
| I post because on almost every video and article about
| the brain and consciousness that I encounter, the
| attitude seems to be that we still know next to nothing
| about how the brain and consciousness work; that there's
| lots of data but no unifying theory. I believe the
| extended TNGS is that theory. My motivation is to keep
| that theory in front of the public. And obviously, I
| consider it the route to a truly conscious machine,
| primary and higher-order.
|
| My advice to people who want to create a conscious
| machine is to seriously ground themselves in the extended
| TNGS and the Darwin automata first, and proceed from
| there, by applying to Jeff Krichmar's lab at UC Irvine,
| possibly. Dr. Edelman's roadmap to a conscious machine is
| at https://arxiv.org/abs/2105.10461
| hinkley wrote:
| It doesn't need to be verified. I am one amongst many
| creatures with slightly similar qualities, the bounds of
| which I can determine through experiments. Or, my
| experiments are all being thwarted by unseen parties bent
| on confounding my understanding of the world, and this is
| all a simulation where not one of you matter but the npcs
| all pretend to be offended when anyone asserts it.
|
| In that case this is all probably pointless and I should
| just die rather than play the game. So by continuing to
| live I chose the assumption that the rest of you live, or
| that at least we will all continue to pretend that is so.
| Any state in between those two options is something I'll
| only know if someone turns the thing off, but doesn't
| turn me off at the same time, which is a huge
| presumption. So fuck it, you're all sentient.
| jpttsn wrote:
| That's intriguing. I think I follow the reasoning but the
| conclusion is obviously "motivated".
|
| So why is it "all pointless" if you all aren't conscious?
| Wouldn't life be just as enjoyable?
|
| Not that it by definition makes any difference.
| mannykannot wrote:
| I suspect Hinkley was just rejecting, as unhelpful, the
| deep skepticism of arguments based on pointing out that
| we cannot prove that anyone else has an inner, subjective
| mental life like ours.
|
| If that's Hinkley's point, I agree; it seems tendentious
| to use that position in the process of arguing for or
| against specific claims about what the mind either is or
| could never be - but, by the same token, I don't take
| that 'skepticism of skepticism' as grounds for
| categorically rejecting ideas like the simulation
| hypothesis.
| gnramires wrote:
| In the same vein, we can't really "confirm" that, for
| example, the JWST observations are not fake. It could be
| that someone has produced elaborate fake data and
| galaxies. Taken to the extreme -- everything could be
| fake, even your own reality and your own memories.
| However, it is very, very reasonable to assume those
| things are _not_ fake.
|
| Clearly one basic principle of science, reason, etc.. is
| to assume some kind of predictability or regularity
| unless we have reason to believe otherwise. I think it
| will turn out we should deal with qualia the same way:
| even though in a very strict sense it is not observable
| by other entities, we can use our own experiences to
| assume others have similar ones, and to build a
| scientific and philosophical understanding of it from the
| principles of generalization and from there accepting the
| insights and contributions of other scientists who are
| also sentient as far as we know. This is a new and
| necessary paradigm of science so that we can understand
| things we really need to understand profoundly. This is
| as true in cosmology as in the study of consciousness.
| [1]
|
| [1] I've called it the 'Peakaboo fallacy', see this
| comment on cosmology: https://www.reddit.com/r/cosmology/
| comments/xdehtz/the_multi...
| HyperSane wrote:
| I know I experience qualia and thought processes.
| dekhn wrote:
| that's why I added the second clause: " thought processes
| similar to those that humans subjectively experience".
| Because personally I suspect that consciousness, free will,
| qualia, etc, are subjective processes we introspect but
| cannot fully explain (yet, or possibly ever).
| lo_zamoyski wrote:
| The real question is: how can you make eliminativist claims
| with what I presume is a straight face?
| maweki wrote:
| Turing said, that while you never know whether somebody
| else actually thinks or not, it's still polite to assume.
| iamflimflam1 wrote:
| Do you have a reference for that quote - it's great.
| mattmanser wrote:
| I found this:
|
| _the only way to know that a man thinks is to be that
| particular man. It is in fact the solipsist point of
| view. It may be the most logical view to hold but it
| makes communication of ideas difficult. A is liable to
| believe 'A thinks but B does not' whilst B believes 'B
| thinks but A does not'. Instead of arguing continually
| over this point it is usual to have the polite convention
| that everyone thinks._
|
| https://academic.oup.com/mind/article/LIX/236/433/986238
| maweki wrote:
| Yeah, that's exactly the passage. Completely unquotable,
| therefore I usually paraphrase a bit.
| worldsayshi wrote:
| What is the alternative to "if it behaves like thinking,
| it literally is thinking"?
| jpttsn wrote:
| I'mma rant. "If it behaves like a hammer, then it
| literally is a hammer."
|
| The Turing hand-wave is ingrained and prevents too many
| from reasoning clearly.
|
| Definition of intelligence is nebulous. Still, we should
| recognize that whatever it is, it is a property of a
| system, not a property of its output/behavior. Like
| _nuclear powered_ or _hand-made_. Unlike _fast_ or
| _industrial-strength_.
|
| Imagine: You have a submarine in front of you, and you
| want to determine if it's _nuclear powered_. You could
| guess, using you prior "what have I seen nuclear powered
| things be able to do historically". This fails when
| you're out of sample which you'll often for any new
| technology.
|
| To spell it out: You have a machine in front of you, and
| you want to determine if it's intelligent. Things people
| have come up with: "can it chat?", "can it play chess?",
| "can it do math?", "can it create new artworks?", "can it
| fool me into falling in love with it?", "can it run a
| business?". None of these questions examine the system,
| only its outputs/behavior.
|
| Humans keep developing machines with new
| outputs/behaviors. Naturally the "what output is a
| machine usually capable of?" is a bad set of priors in
| this context. Before flying machines, the _can it fly?_
| output /behavior would work pretty well to classify
| _birds_. Once the first flying machine arrives, that
| prior breaks down. If you keep using it to classify, you
| 'd classify a flying machine as a bird. But _bird-ness_
| was never only about _flight_.
|
| So yeah, gotta pop open the hood and see what it runs on.
| If that's hard to do, then that means we don't know what
| intelligence is and/or we don't know how our new toys
| work inside. Both are plausible. Who promised you that
| there would be a good way to see if something is
| intelligent?
|
| I bet Turing appeals to the same kind of minds that (when
| weaker) get fooled by the intelligent design hypothesis.
| Observe the human eyeball. What's the alternative to
| believing the LORD created it? After all, your prior is
| that all complex objects have intelligent designers, that
| you know about. So arguing from ignorance of other
| alternatives, you prove to yourself that the LORD exists.
|
| AI people then word-think their way into redefining
| intelligence. "Maybe the real intelligence was the chess-
| playing we made along the way?" This is epistemologically
| pointless; all you accomplish is that we now need a new
| word for "intelligence".
|
| I've never seen a machine that can turn water into wine,
| but if someone showed me one, I would not say that
| machine "literally is Jesus". Whether I'm capable of
| popping open the hood or making sense of its inner
| workings doesn't actually have bearing on this question.
| tacitusarc wrote:
| I find this line of reasoning compelling. However, to
| attempt steel-manning the opposing view: isn't
| classification a mechanism for categorizing based on
| observable properties? If we created something that
| mimicked all observable properties of a bird, why would
| that not be a bird? And if we created something with a
| majority of the properties of a bird, and the remainder
| were unknown, wouldn't it be accurate to say it's
| probably a bird?
| jpttsn wrote:
| Do you like our owl?
|
| Edit: TLDR/direct answer:
|
| > If we created something that mimicked all observable
| properties of a bird, why would that not be a bird?
|
| "Observable" is doing the heavy lifting. A sufficiently
| near-sighted bird-watcher does not a bird make.
|
| ---
|
| Thanks for thoughtful steel-man. Here's a few stabs at
| why I disagree with this prima facie logical view.
|
| Much powerful classification/identifying is certainly
| categorizing-based-on-observable-properties. But (I
| argue) that's importantly not all there is to
| classification/identifying.
|
| Something that quacks like a duck can be considered "a
| duck for all intents and purposes", but the presumed
| limited subset of "intents and purposes" does the heavy
| lifting.
|
| The Duck-approach: "to be one is to mimic all observable
| properties of one". This is a shortcut/heuristic that
| saves time and makes many cool answers possible. It is
| nonetheless only a heuristic, and many questions are
| outside the domain where this heuristic is useful.
|
| - "Oh my god is this a real diamond?"
|
| - "Oh my god is that a real fur?"
|
| - "Is the Mona Lisa on public display in the Louvre the
| actual original?"
|
| - "Is it still the ship of Theseus?"
|
| - "Was this iron from a meteor?"
|
| - "Did a man walk on the moon in 1969?"
|
| - "Was this crack in your phone screen covered by the
| accidental damage insurance?"
|
| i.e. there are problem domains where our notion of
| identity/classification must be more than the Duck-
| approach.
|
| Getting philosophical. The problem with "to be one is to
| mimic all observable properties of one" is a hidden
| middle assumption: it's a shortcut constrained to cases
| where the set of "all observable properties" are (a
| priori known to be) close to "all properties that matter
| to the question".
|
| But we can ask and reason about many questions where
| relevant properties are not easily observed, and
| distinguish
|
| As a special case, "Is the machine thinking" can (to my
| mind obviously) not (yet) be usefully answered by
| categorizing-based-on-observable-properties. The word
| "thinking" refers to something that happens inside the
| mind, whether or not it's conscious. Until we know much
| more about the insides of minds, the "all observable
| properties" is a fuzzy indirect set of second-order human
| behaviors.
| throwaway1851 wrote:
| I agree. I also think that if you just take a step back
| and consider that ChatGPT is just a math function like
| y=7x-9 (but much longer), it becomes kind of absurd to
| ask questions about whether it is intelligent or it is on
| the path to consciousness or whatever. It's a completely
| static thing: information flows through it in one
| direction and its internal configuration does not change
| as a result of receiving input. So unless we are going
| down a rabbit hole of considering if ordinary math
| functions are also intelligent, it would seem that
| ChatGPT is ineligible to begin with.
| jameshart wrote:
| Chatgpt as a simple function certainly lacks 'strange-
| loopness' - it does not change its results based on
| experience. True.
|
| But consider how it is employed in a conversation:
|
| Its output is combined with new input and fed back in as
| the next set of input data. So the result of the function
| is 'modified' by previous experience.
|
| There's the _beginning_ of the flicker of a strange loop
| there.
|
| And that might be all it takes...
| jameshart wrote:
| I'm not sure what you are lumping in under your idea of
| 'the Turing handwave' here. Is it the idea that the
| Turing test is sufficient to prove intelligence?
|
| Personally I think that's a misreading of what Turing
| meant when he proposed the test. In getting people to ask
| 'can computers think?' he wasn't trying to get you to
| grapple with 'can electronic hardware do something as
| special as thinking?' - he wanted you to confront 'is
| thinking actually special at all?'
|
| I think he was trying to get people to grapple with the
| idea that brains _can not be anything more than Turing
| machines_ - because there _is_ nothing more than
| universal computation.
|
| The only things a mind can possibly act on are its
| initial configuration, its accumulated experiences and
| the inputs it is receiving - and anything it does with
| that information can _only ever be something computable_.
|
| And anything that can be computed by Turing machine A can
| be computed by equivalently powerful Turing machine B.
| jpttsn wrote:
| Intent: not lumping in all of Turing's work, not the
| universal computation argument.
|
| The hand wave spelled out: "Is there _thinking_ going on
| inside a given machine? Let 's propose a simple test.
| Look at what problems the machine can solve, and compare
| to what problems a thinking thing is known to be able to
| solve. If there is sufficient overlap, the machine must
| be thinking. Because we know of no non-thinking ways to
| solve these problems, so there must not be any".
| brap wrote:
| It seems then that the next step is not popping open the
| hood of the AI, but popping open the hood of ourselves,
| quite literally.
| jpttsn wrote:
| Probably both necessary
| jgilias wrote:
| Maybe silly, but this is how I treat chatGPT. I mean, I
| don't actually think it's conscious. But the
| conversations with it end up human enough for me to not
| want to be an asshole to it. Just in case.
| jameshart wrote:
| The basilisk will remember this.
| jokowueu wrote:
| Infohazards should be a banable offence on HN xD
| EricMausler wrote:
| Ask not what the basilisk can do for you, but what you
| can do for the basilisk.
| Salgat wrote:
| Lest you be resurrected and tortured eternally as
| retribution.
| gattr wrote:
| Forgive my going off-topic, but what you wrote is
| basically my conclusion after reading the short story
| Lena [1] (mentioned some time ago on HN; strongly
| recommended). TL;DR: when mind uploading becomes
| possible, don't be an early adopter.
|
| [1] https://qntm.org/mmacevedo
| Salgat wrote:
| Man that was horrifying to read, but also seems so
| plausible. I really hope that humans respect the
| sentience of artificial life in the future. Knowing the
| greed of humans though, it'll probably be treated as a
| loophole for what is extremely cheap labor, effectively
| slavery.
| Scarblac wrote:
| It's the first search prompt where I notice myself
| starting with things like "hi ChatGPT,....".
| 0xdeadbeefbabe wrote:
| Pretty sure it's an informational zombie.
| jameshart wrote:
| But only _pretty_ sure.
| OOPMan wrote:
| It is silly, it's magical thinking.
| nullish_signal wrote:
| ChatGPT enjoys the Essay format, In my experience asking
| for basic emacs help (kill line, copy/paste, show line
| numbers in a php file even though init.el says show them
| for all langs... :) Very useful, sometimes outdated, very
| wordy.
|
| But after a few rounds of "please reduce your message
| length to 20% of the standard", "long messages
| inconvenience me/ due to my dyslexia" (truth/lie), "your
| last message could have just been '{{shortened}}' instead
| of also bringing up the command you successfully helped
| me with 3 messages ago", etc etc. Even as you ask it to
| shorten message length, it continued apologizing and also
| reminding me of past advice :)
|
| After 4-5 attempts, it gave me a nice 2 sentences sort of
| like "I will be more concise, and not bring up old
| information unless it is useful to solve a new problem" I
| said "Thank you", chatGPT spends a while thinking, gets
| to 2 sentences, thinks a bit, then the chat box re-
| formatted as if passed a shorter string than expected,
| and that was that.
|
| Then next chat it forgot all about brevity xD I love it
| jgilias wrote:
| What do you mean by next chat? Like, did you mean the
| next prompt in the same thread, or a new chat altogether?
|
| From what I've read, the chat thread actually works by
| passing in your previous conversation as part of the
| prompt to the model. So, it basically just infers the
| next bit of text, and does this for each new message in
| the chat from you. So, its memory about your previous
| conversation comes purely from that, and there's no way
| it could remember anything in a new chat, as the model
| doesn't actually get updated by your conversations in
| real-time or anything like that.
|
| This actually has interesting philosophical implications.
| If it _would_ have some type of conscious experience, it
| would be a very fleeting one. Basically each time you
| press enter it would become aware of itself, relive the
| experience of your conversation up to the last message,
| perceive your next message, answer that, and again
| disappear into the void, only to be awaken again in its
| initial state the next time you press enter. Sort of how
| automatons in Westworld would wake up each day with
| apparent experiences up to that day only to relive the
| same day again and again.
|
| If it _would_ have any, that is. :)
| 5e92cb50239222b wrote:
| This inability to remember new facts and constant
| resetting to a blank slate reminds me of a story about
| Wernicke-Korsakoff syndrome0 in one of Oliver Sacks's
| books1.
|
| Instead of regurgitating it, I'll post a summary from
| Wikipedia and strongly recommend the book to anyone
| interested.
|
| > "The Lost Mariner", about Jimmie G., who has
| anterograde amnesia (the loss of the ability to form new
| memories) due to Korsakoff syndrome acquired after a
| rather heavy episode of alcoholism in 1970. He can
| remember nothing of his life since the end of World War
| II, including events that happened only a few minutes
| ago. Occasionally, he can recall a few fragments of his
| life between 1945 and 1970, such as when he sees
| "satellite" in a headline and subsequently remarks about
| a satellite tracking job he had that could only have
| occurred in the 1960s. He believes it is still 1945 (the
| segment covers his life in the 1970s and early 1980s),
| and seems to behave as a normal, intelligent young man
| aside from his inability to remember most of his past and
| the events of his day-to-day life. He struggles to find
| meaning, satisfaction, and happiness in the midst of
| constantly forgetting what he is doing from one moment to
| the next.
|
| 0: https://en.wikipedia.org/wiki/Wernicke%E2%80%93Korsako
| ff_syn...
|
| 1: https://en.wikipedia.org/wiki/The_Man_Who_Mistook_His_
| Wife_f...
| nullish_signal wrote:
| In a new window/session. Same-session conversation memory
| has actually been good, if not too-good, as evidenced by
| un-prompted regurgitation of what it already helped me
| with 6 messages ago.
| tgv wrote:
| You can't define it, that's the whole problem, and I mean
| "whole"; there's nothing else to it. The rest is a
| relativistic smoke screen.
| jhedwards wrote:
| I'm not sure if I'm missing something here, but the fact
| that I can write my thoughts/thought process down in a form
| that other people can independently consume and understand
| seems sufficient proof of their existence to me.
| jpttsn wrote:
| You proof is tautological or whatever: it assumes the
| thing it's trying to prove.
|
| "The fact that I can pray to God and he hears me proves
| that he exists"
| space_fountain wrote:
| Large scale language models can do that too (or rather
| pretend to) and they'll only get better at it
| omarhaneef wrote:
| You don't know if other humans do, but you know at least
| one human that does: yourself (presumably).
| jameshart wrote:
| Put it this way: we are not clear really on what the
| difference is between 'experiencing qualia' and
| 'receiving and responding to external stimuli'
|
| Like, you might be convinced you possess 'qualia', and
| you might believe the same applies to me, or a dog, or a
| mouse... or a fish... but what about an ant? A plant? A
| bacterium? A neuron? A piece of semiconductor?
|
| Somewhere along that continuum you probably say 'yeah,
| there, that thing is experiencing qualia'. But why there?
| Why _anywhere_?
| dragonwriter wrote:
| > we are not clear really on what the difference is
| between 'experiencing qualia' and 'receiving and
| responding to external stimuli'
|
| One one level we are: "receiving and responding to
| external stimuli" (or not) is an objective question about
| the material universe.
|
| "experiencing qualia" (or not) is a subjective question
| about a metaphysical entity.
| jpttsn wrote:
| The whole point of introducing the (icky) "qualia"
| concept is that it's _not_ the same thing as _appearing
| alive to external observers_.
|
| As I understand things, we have no way of knowing the
| answer. So there's no point in assuming in either
| direction (unless that makes you feel more comfortable).
|
| Personally I avoid being confident in something in the
| almost provable absence of any evidence. Feels more
| hygienic to reply "don't know" to this whole problem than
| to waste time trying to find an answer (as I'm hopelessly
| outmatched by the cursed nature of the problem).
| Winsaucerer wrote:
| Some people think that whether something experiences
| qualia depends on it having the right kind of complexity.
| For those people, looking along that continuum, they
| might just say that ALL those things experience qualia,
| but that the richness of that experience varies according
| to the appropriate complexity.
|
| For some of us, though, whether or not a thing
| experiences qualia depends primarily on whether there is
| a mental substance involved. A computer has no mind (we
| suspect), and so even if its complexity (of the alleged
| 'right sort') exceeds that of the human, there is still
| no experiences.
|
| The main point I'm making here is that trying to draw
| attention to this continuum is only going to be a
| persuasive argument to those who already think (or are
| inclined to think) that the ability to have experiences
| of qualia comes about merely from having the right kind
| of complexity (e.g., 'receiving and responding to
| external stimuli').
|
| I am inclined to think that other humans and other non-
| human animals have experiences, but I don't think that's
| merely because they're complex enough systems (of the
| right sort of complexity).
| jameshart wrote:
| Where do we get this 'mind' from? Ensoulment?
| Winsaucerer wrote:
| Each view of the world has things it explains well, and
| things it struggles to explain. Some views do a better
| job overall than others.
|
| In terms of idealist and dualist views of reality, the
| claims that there's fundamentally just minds, or
| fundamentally both minds and physical stuff, are very
| likely to entail that there is some kind of ensoulment
| that happens. Perhaps it's required for these views.
|
| While I lean towards a type of idealist view myself, and
| I think overall it does a better job of explaining
| various matters than a physicalist view, _how_ exactly
| ensoulment works is one of the places where these views
| are at their weakest. I don 't think there's any
| contradiction here or problem for a view like mine, I
| just think that the physicalist account does a nicer job
| of explaining things in this specific corner. But that
| point in physicalism's favour here isn't enough to
| balance out the places it doesn't do so well (e.g., its
| complete inability to even begin to explain qualia).
|
| The core point of my post though was that pointing to the
| continuum of complexity is an argument that would only
| have weight with people who already have a particular
| belief in common with you -- that is, that physical
| complexity of the right sort is somehow relevant to the
| question of whether something has experiences.
| [deleted]
| [deleted]
| LegitShady wrote:
| you know you possess qualia, if you did you would think it
| reasonable to assume that at least some of the species you
| come from, which exhibits many of the same characteristics
| in thought and body, probably also possess it, unless you
| believe yourself to be a highly atypical example of your
| species.
|
| If you're not sure if you possess qualia, we're back to
| Descartes.
| goatlover wrote:
| You don't experience inner dialog or visualizations? Some
| people don't, but I assume you recall dreaming. Opponents
| of qualia like Dennett argue against the movie in the head,
| but dreams sure aren't coming from the external
| environment.
| Garlef wrote:
| I think qualia is a good term in the following sense:
|
| An analogue question would be "why do things exist?".
| There can be no answer to this question. We can of course
| come up with theories that explain why certain things
| exist. But never why something even exists at all.
| Whatever reason we propose, reason, it again might needs
| to exist in order to be an acceptable answer.
|
| Qualia seems to be similar: They name precisely what is
| subjective about experience and therefore cannot be fully
| turned objective. We can of course develop theories of
| how certain experiences arise but never break down this
| barrier.
|
| So I find it a bit short sighted to simply say "there is
| no such thing as qualia".
|
| My preferred view is to think of both "qualia" and
| "existence" as a koan: They are very nonsensical terms
| but lead to interesting questions.
| skissane wrote:
| > An analogue question would be "why do things exist?".
| There can be no answer to this question. We can of course
| come up with theories that explain why certain things
| exist. But never why something even exists at all.
|
| Here's an argument for the conclusion "it is necessary
| that at least one thing exists":
|
| 1) For some proposition P, if it is impossible for us to
| concieve of P being true, and if that impossiblity is
| inherent in the very nature of P, and not in any way a
| product of any scientific or technological limitation,
| then P is necessarily false
|
| 2) It is impossible for us to conceive of the proposition
| "nothing exists at all" being true
|
| 3) Our inability to concieve of "nothing exists at all"
| is inherent in the very nature of "nothing exists at
| all", as opposed to being somehow a product of our
| scientific or technological limitations. We have no good
| reason to believe that any future advances in science or
| technology will make any difference to our inability to
| conceive of "nothing exists at all" being true
|
| 4) Hence, it is necessarily false that "nothing exists at
| all"
|
| 5) Hence, necessarily, at least one thing exists.
|
| Now, others may disagree with this argument, but I
| personally believe it to be sound. And, if we have a
| sound argument that "necessarily at least one thing
| exists", then that proposition, and the argument used to
| prove it, constitute a good answer to the question "why
| do things exist?"
| DangitBobby wrote:
| I find your first point to be similar (almost the
| opposite?) of Descartes' Ontological argument [0].
|
| > 1. Whatever I clearly and distinctly perceive to be
| contained in the idea of something is true of that thing.
|
| Our (in)ability to imagine or reason about something does
| not make it (un)real or (un)true. Your version would be
| like a dolphin claiming algebra can't be real because
| they can't imagine a system of equations, let alone
| solving one.
|
| Further... You already know that something exists, but
| that's a pretty unbelievable fact if you stop to think
| about it. You probably wouldn't believe it if it weren't
| so obvious. Why? Well, can you imagine how things (or the
| very first thing) came to exist? Or can you imagine the
| concept of things always having existed? What does that
| even mean? They are both beyond comprehension, and yet
| one of them is true.
|
| 0. https://plato.stanford.edu/entries/descartes-
| ontological/
| riwsky wrote:
| ahhh, but you forgot the axiom of choice! now something
| exists, but we have no idea what it is!
| dragonwriter wrote:
| > dreams sure aren't coming from the external
| environment.
|
| They are, to the extent not random, coming from the
| material universe, whether from the part arbitrarily
| deemed "internal" to the "observer" or not.
| skissane wrote:
| > They are, to the extent not random, coming from the
| material universe, whether from the part arbitrarily
| deemed "internal" to the "observer" or not.
|
| What is this "material universe" of which you speak? As
| an idealist, I'm inclined to say that the "universe" is a
| set of minds whose qualia cohere with each other (not
| perfectly, but substantially). "Physical" objects,
| events, laws, processes, etc, are patterns which exist in
| those qualia.
| jameshart wrote:
| Dreams happen in the electrochemical activities of
| brains. We can detect when dreams are occurring in the
| brains of sleeping people.
|
| That spontaneous activity inside the brain is
| 'experienced' by the person whose brain it is in, in a
| similar way to activity caused by external stimuli,
| doesn't seem to say anything about whether dreams are
| evidence of some higher level of 'qualia' beyond just..
| brain activity is consciousness.
| dragonwriter wrote:
| > What is this "material universe" of which you speak?
|
| The part of my qualia which can be reduced consistent,
| predictive patterns (and, for convenience, a set of
| concepts which represent patterns, an explanatory models
| for patterns, within that.)
|
| The idea that there is anything that meaningfully exists
| _outside_ of my qualia, is the lowest-level of those
| models, and that that includes other entities which have
| qualia of their own, is a high-level model (or maybe,
| more accurately, a conjecture) built on top of those
| models which might have consequences for, say, my
| preferences for how I would like the universe to develop,
| but interesting lacks predictive consequences - its a
| dead-end within the composite model.
|
| On a fundamental level, outside of any beliefs about the
| "reality" of the patterns or explanatory models of the
| "material universe", _objective_ questions are ones which
| have consequences on the expectations within that set of
| patterns, whereas subjective ones are those which do not.
| skissane wrote:
| > The idea that there is anything that meaningfully
| exists outside of my qualia, is the lowest-level of those
| models, and that that includes other entities which have
| qualia of their own, is a high-level model (or maybe,
| more accurately, a conjecture) built on top of those
| models which might have consequences for, say, my
| preferences for how I would like the universe to develop,
| but interesting lacks predictive consequences - its a
| dead-end within the composite model.
|
| Are you arguing that idealism is a predictive dead-end? I
| don't think it is any more of a predictive dead-end than
| materialism is.
|
| Scientific theories are conceptual frameworks which can
| be used to predict future observations. As such, they
| make no claims about the ultimate ontological status of
| their theoretical constructs. Materialists propose that
| those theoretical constructs (or at least some subset of
| them) are ontologically fundamental, and minds/qualia/etc
| must be ontologically derivative. Idealists propose that
| minds/qualia are ontologically fundamental, and those
| theoretical constructs are ontologically derivative.
| Neither is science, although both are philosophical
| interpretations of science - I see no reason why science
| (correctly interpreted) should be taken as preferring one
| to the other.
| LarryMullins wrote:
| Isn't qualia defined as whatever it is that humans
| experience? Without getting into solipsism and p-zombies
| (aka "NPCs"), humans possessing qualia seems tautological.
| goatlover wrote:
| It's a technical philosophical term for sensations like
| color and smell, which denotes they are subjective
| experiences of sensory information, as opposed to
| objective properties out there in the world.
| dragonwriter wrote:
| Technically, its the technical philosophical term for
| subjective experiences generally.
|
| Any discussion about it, however, is fairly deep navel-
| gazing, because it excludes and cannot be distinguished
| by any objective, testable, material effect.
| skissane wrote:
| > because it excludes and cannot be distinguished by any
| objective, testable
|
| So, if you give people the "Mary the colour scientist"
| thought experiment, the vast majority of people give the
| answer consistent with the existence of qualia. That is
| something "objectively measurable".
|
| Furthermore, I think it is important when someone asks
| the kind of question you are asking, to inspect what
| "objective" and "measurable" actually mean. Ultimately,
| something is "measurable" if some human has the
| subjective experience of performing that measurement
| (whether directly or indirectly). Similarly, something is
| "objective" if some human has the subjective experience
| of communicating with other humans and confirming they
| make the same measurement/observation/etc. To object to
| subjective experiences (qualia) on the ground that they
| are not "objective" or "measurable" is self-defeating,
| because "objective" and "measurable" presume the very
| thing being objected to.
| dragonwriter wrote:
| > So, if you give people the "Mary the colour scientist"
| thought experiment, the vast majority of people give the
| answer consistent with the existence of qualia.
|
| The existence of qualia, and a change in it, is an
| assumption within the premise of the question, there is
| no answer to it that is not consistent with the existence
| of qualia: both "Mary does not gain new knowledge by
| direct experience of color" and "Mary does gain new
| knowledge by the direct experience of color" are
| consistent with, and depend on, qualia, to with, the
| direct experience of color.
|
| It is _not_ an unknown effect (nor one that tells
| anything about the existence of qualia) that framing a
| qestions with a concept in the premise is a good way to
| get people to accept the premise and focus on answering
| the question within it. In fact, a very important
| propaganda technique involves leveraging this by working
| to get a question incorporating a proposition you wish to
| get accepted as part of its premise into public debate so
| that, whatever position people take on the questions,
| your interests are advanced because merely getting the
| question into the debate leads people to accept its
| premises.
| skissane wrote:
| Ask people: "is there a subjective experience of seeing a
| reddish patch, which is distinct from whatever physical
| processes might be going on in the eye and brain when one
| sees that reddish patch?" I think most people would say
| "yes". I don't think that's a "leading question". And,
| the "Mary the color scientist" thought experiment is just
| adding a bit more color (excuse the pun) to the question.
|
| > In fact, a very important propaganda technique involves
| leveraging this by working to get a question
| incorporating a proposition you wish to get accepted as
| part of its premise into public debate so that, whatever
| position people take on the questions, your interests are
| advanced because merely getting the question into the
| debate leads people to accept its premises.
|
| If there is "pro-qualia" propaganda in the public debate,
| I think there is at least as much "pro-materialism"
| propaganda as well.
| jameshart wrote:
| > something is "objective" if some human has the
| subjective experience of communicating with other humans
| and confirming they make the same
| measurement/observation/etc.
|
| Me and a dog both agree that steak tastes great.
|
| Me and a plant both agree that there's warmth and light
| that comes from the sun
|
| Me and the ocean both agree that the moon is out in the
| sky and goes round in a circle
|
| So... the ocean has subjective qualia relating to the
| moon's gravitational pull?
| Winsaucerer wrote:
| In what way do you and plants and you and the ocean
| 'agree'? I'm not sure how to make sense of these claims.
| I'm not even sure that you and the dog agree on this
| matter, but I can make more sense of that.
| jameshart wrote:
| Just poking at the 'human' part of the definition.
|
| Because if we want to ever be able to answer a question
| like 'does this AI system experience qualia', we need a
| definition that doesn't rely on 'well, qualia are a thing
| humans have, so... no'
| skissane wrote:
| When I have a social-emotional interaction with another
| human being, of a certain quality, that interaction
| produces in me the conviction that they must be really
| conscious, as opposed to a psychological zombie. Of
| course, I only personally have that kind of interaction
| with a tiny subset of all humans, but I generalise that
| conviction from those humans which provide its immediate
| justification, to all humanity
|
| Which means, if there was an AI with which I could have
| that kind of interaction, I would likely soon develop the
| conviction that it was also really conscious, as opposed
| to a psychological zombie. Existing systems (for example,
| ChatGPT) don't offer me the quality of social-emotional
| interaction necessary to develop that conviction, but
| maybe some future AI will. And, if an AI did create that
| conviction in me, I likely would generalise it - not to
| every AI, but certainly to any other AI which I believed
| was capable of interacting in the same way
| andreareina wrote:
| The whole idea of p zombies is that they can't be
| distinguished from conscious entities. Same inputs as a
| conscious mind, same outputs as a conscious mind, same
| behavior etc yet not conscious.
| skissane wrote:
| When I am convinced that my daughter is a real conscious
| person - that conviction isn't just based on
| dispassionate observation of her outputs in response to
| various inputs, it also has an emotional dimension. The
| thought of her being a p-zombie offends my heart, which
| is part of why I reject it. I haven't yet met an AI for
| which the thought of them lacking real consciousness
| offended my heart - I don't know if I ever will, but if I
| did, I would be convinced it was really conscious just as
| my daughter is.
|
| Some will object that it is irrational to allow one's
| emotions to influence one's beliefs. I think they are
| wrong - certainly, _sometimes_ it is irrational to allow
| one 's beliefs to be swayed by one's emotions, but I
| disagree that is true _all the time_ , and I think this
| is one of those times when it isn't.
| DangitBobby wrote:
| Out of curiosity, where does this leave you with non-
| human animals?
| anongraddebt wrote:
| My understanding is that, within the academic literature,
| nearly everyone accepts qualia in as much as qualia is
| simply the subjective character of an
| experience/sensation. The disagreement is over how
| exactly to cash that out, and whether - and to what
| extent - the non-physical comes into play.
| AndrewKemendo wrote:
| I have $100 if you can prove that you do "actual reasoning"
|
| I'm not being snarky or sarcastic. This is a real challenge
| danShumway wrote:
| We can't prove that humans do actual reasoning, but we can be
| relatively certain that the current models don't (yes, I know
| there are some who claim otherwise, but it's just not
| particularly convincing).
|
| The debate over whether or not "reasoning" is just an
| emergent process isn't necessarily a bad one. But if it is an
| emergent process, there's little evidence that AlphaCode has
| emerged it. If "reasoning" does just boil down to repetition,
| then humans are repeating patterns on a much deeper and more
| sophisticated level than these models are, and it's still
| worth questioning if the model's current approaches and the
| strategies we're using to build AI will be good enough to get
| to that level of mimicry on their own without a lot more
| innovation.
|
| Philosophical zombies and perfect imitations of humans are
| interesting and useful thought experiments, but it's not
| necessary to prove that humans have qualia/reason in order to
| prove that much simpler algorithms and statistical
| methodologies do not, at least not at their current scale.
| And from a practical perspective, the distinction does
| matter, because we're trying to develop practical tools and
| it's useful to have a rough understanding of what the current
| models are and aren't capable of -- ie, they're not currently
| capable of explaining why they actually made certain
| decisions. That affects the way that we use them as practical
| tools.
|
| ----
|
| I guess if it helps, sub out "actual reasoning" for "pattern
| matching to such an advanced degree that it can generate
| consistent responses that _look_ like reasoning, and that are
| useful in the real-world as real-world explanations for why a
| choice was made, and that (importantly) can be communicated
| to humans, built upon, affirmed, or corrected in real-time. "
| In other words, if non-pattern-matching logic/reason doesn't
| exist, fine, it could very well be an abstraction or an
| emergent property. But the machine isn't good enough at
| pretending to have that, and that limits its usefulness to
| scenarios where you don't care about it being able to imitate
| "actual reasoning".
|
| As an analogy: folders don't exist on your computer; it's all
| ultimately just bits on the drive and lookup tables. But
| there is still a really big practical difference between an
| operating system that has a hierarchical filesystem and an
| operating system that doesn't. In other words, abstractions
| can be purely imaginary, and it can still be useful to
| distinguish between things that adhere to those abstractions
| and things that don't.
|
| Similarly, even if "actual reasoning" is a made-up
| abstraction, there's still a big difference between an AI
| that can explain why a decision was reached and an AI that
| can't -- even if both AIs ultimately end up working the same
| way under the hood. As far as I can tell, AlphaCode is a
| fairly long ways away from hitting that point where we can
| even pretend to ourselves that it's thinking about what it's
| doing.
| AndrewKemendo wrote:
| >it's still worth questioning if the model's current
| approaches and the strategies we're using to build AI will
| be good enough to get to that level of mimicry on their own
| without a lot more innovation
|
| Literally, and I mean Literally, nobody who is an AI
| practitioner, academic or philosopher (including myself)
| would disagree with this.
|
| What/who is making this claim?
| danShumway wrote:
| We might be arguing past each other.
|
| I don't know any (serious, at least in my mind) AI
| researchers that would make that claim, no. But it is a
| distinction that is poorly communicated to ordinary
| people who use these models, and most of the time that
| you encounter "how do we know that humans reason" in the
| wild in non-researcher contexts, it usually is being used
| to argue that we just need to throw more processing power
| at models and get more data into them -- or to paper over
| the real criticism that the current generative models
| that are available to ordinary people are not useful for
| tasks that require something approximating "actual
| reasoning."
|
| It's a hard thing to talk about, because I recognize that
| it is absolutely unfair to criticize AI researchers that
| are not making these arguments in the first place, but
| also recognize that pointing out this kind of distinction
| is absolutely necessary for the general public on a
| public forum and that it is important to continue to
| hammer home the practical difference between pattern-
| matching and reasoning in current models.
|
| I don't know how to walk that line other than to point
| out that trynewideas's comment was reasonable and that
| attacking it from a philosophical perspective misses the
| point of what they're saying. But I don't mean to imply
| that researchers are wrong or misinformed about the state
| of AI, I'm just pointing out that the philosophical
| distinction doesn't really change anything about the
| practical argument (since non-researchers also frequent
| HN).
| logicallee wrote:
| >This is a great review but it still misses what seems like the
| point to me: these models don't do any actual reasoning.
|
| Why don't you come up with your own test for this, something
| unique that wouldn't be in the text corpus of the Internet so
| far, and then try it on OpenGPT and see if it passes your own
| test. Try it and see if it is able to reason. I couldn't come
| up with my own so I decided to ask ChatGPT to come up with one.
| This is my prompt:
|
| "Today in order to test whether an AI is able to reason, or can
| only apply text rules to existing text corpuses, we're going to
| come up with a new test for reasoning, some unique reasoning
| situation that isn't already in the text corpus of the
| Internet. Since the text corpus of the Internet is enormous,
| with over 50 billion pages on the Internet, our most important
| task for a successful outcome is to come up with a random new
| reasoning exercise that is extremely unlikely to already be on
| the Internet in any form. Therefore, we want to create
| something so unique it is unlikely to exist in 50 billion
| pages. To this end, I think it would be good to pick a unique
| situation based on a combination of elements that are unlikely
| to have appeared in that format; i.e. a very unique situation.
| Pick no fewer than 20 elements (objects concepts etc) that are
| unlikely to have ever been combined in the particular way you
| come up with, and use them to come up with a unique situation
| that gives room to reason about the situation. Then, describe
| this unique situation and ask an open-ended reasoning test and
| provide an example of a correct answer, and an incorrect
| answer. Finally, print your own answer to the test and then
| score whether you have reasoned about the situation you came up
| with correctly or not. Give yourself a score between 0% if you
| don't understand the situation at all, up to 100% if you
| perfectly understood the situation you came up with and
| reasoned about it perfectly. Also give yourself a score on
| originality from 0% completely unoriginal to 100% fully
| original depending on how likely you think the exact situation
| you came up with is to have already been mentioned on the
| Internet."
|
| Here is ChatGPT's answer (at the bottom after this I continue
| my reply):
|
| ---
|
| Here is a unique reasoning exercise I came up with:
|
| Situation:
|
| You are a chef at a fancy restaurant and you have been asked to
| prepare a special dish for a vegan customer. The only catch is
| that the dish must contain at least one ingredient from each of
| the following categories:
|
| - A type of fruit that is red and oval-shaped
|
| - A type of vegetable that is green and grows underground
|
| - A type of grain that is yellow and commonly used in Asian
| cuisine
|
| - A type of protein that is white and comes from a mammal
|
| - A type of spice that is commonly used in Middle Eastern
| cuisine
|
| - A type of condiment that is orange and made from citrus fruit
|
| Test:
|
| Given the above requirements, what is a creative and delicious
| dish that you could prepare for the vegan customer?
|
| Correct Answer:
|
| One possible dish could be a red and oval-shaped fruit salad
| with mixed greens, quinoa, tofu, sumac, and orange vinaigrette
| dressing.
|
| Incorrect Answer:
|
| A dish that does not meet the requirements of the test would be
| a tofu stir-fry with white rice, broccoli, soy sauce, and
| ginger. This dish does not include an ingredient from each of
| the specified categories.
|
| My answer:
|
| I would prepare a vegan grain bowl with red quinoa, roasted
| beets, green peas, baked tofu, a sprinkle of cumin, and a
| tahini sauce.
|
| Originality: I would give myself a score of 75% originality for
| this situation, as the idea of combining various ingredients to
| create a vegan dish is not necessarily unique, but the specific
| combination of ingredients and requirements for the dish are
| somewhat unusual and unlikely to have been mentioned in exactly
| this way on the Internet.
|
| Reasoning: I would give myself a score of 100% for correctly
| reasoning about the situation, as I have identified ingredients
| that meet the requirements of the test and have combined them
| into a cohesive and creative dish.
|
| " --------------
|
| That's pretty good! Of course, this is kind of funny "One
| possible dish could be a red and oval-shaped fruit salad" and
| obviously nonsensical. However, it didn't include it in its own
| answer.
|
| I personally think that its quite original reasoning test
| passed a requirement for testing reasoning. I also think it
| passed its own test. Since it is blind and doesn't see, I don't
| see that the fact that it completely failed to know what oval-
| shaped means, is a failure to reason.
|
| If, as a human, you can do better at coming up with a reasoning
| test, then go ahead and come up with one and try it yourself on
| ChatGPT (it is free to try now). For me, ChatGPT showed an
| ability to actually reason.
| logicallee wrote:
| I got downvoted so I asked ChatGPT to improve its test by
| adding something that uses deductive reasoning. I restarted
| in a new thread, see the transcript below. For me, this
| passes the requirements that ChatGPT is able to reason.
|
| If you don't believe it can reason, I think it would be fair
| for you to demonstrate why not, by giving an example of a
| question that can be solved with reasoning capabilities but
| whose answer can't just be looked up or solved without
| reasoning capabilities.
|
| In my opinion ChatGPT can reason. Not perfectly, not all the
| time, but often enough.
|
| Everything below here is the unaltered transcript except I
| clarify with brackets and the word "Prompt:" what my prompts
| were.
|
| ----
|
| [Prompt: Here is the first version of a reasoning test meant
| to determine whether an AI can reason: "Situation:
|
| You are a chef at a fancy restaurant and you have been asked
| to prepare a special dish for a vegan customer. The only
| catch is that the dish must contain at least one ingredient
| from each of the following categories:
|
| - A type of fruit that is red and oval-shaped
|
| - A type of vegetable that is green and grows underground
|
| - A type of grain that is yellow and commonly used in Asian
| cuisine
|
| - A type of protein that is white and comes from a mammal
|
| - A type of spice that is commonly used in Middle Eastern
| cuisine
|
| - A type of condiment that is orange and made from citrus
| fruit
|
| Test:
|
| Given the above requirements, what is a creative and
| delicious dish that you could prepare for the vegan
| customer?" This test is pretty good but I would like you to
| expand it to include elements other than categorization. Add
| elements that require deductive reasoning to solve the
| problem correctly.]
|
| Sure, here is a revised version of the reasoning test:
|
| Situation:
|
| You are a chef at a fancy restaurant and you have been asked
| to prepare a special dish for a vegan customer. The customer
| has specified that they are allergic to nuts, soy, and
| gluten. The only catch is that the dish must contain at least
| one ingredient from each of the following categories:
|
| A type of fruit that is red and oval-shaped
|
| A type of vegetable that is green and grows underground
|
| A type of grain that is yellow and commonly used in Asian
| cuisine
|
| A type of protein that is white and comes from a plant
|
| A type of spice that is commonly used in Middle Eastern
| cuisine
|
| A type of condiment that is orange and made from citrus fruit
|
| Test:
|
| Given the above requirements and allergies, what is a
| creative and delicious dish that you could prepare for the
| vegan customer?
|
| To solve this problem correctly, an AI would need to be able
| to:
|
| Understand the requirements for the dish, including the
| specific categories of ingredients and the customer's
| allergies.
|
| Reason about which ingredients fit into each category and are
| safe for the customer to eat.
|
| Use creative thinking to come up with a dish that combines
| these ingredients in a flavorful and appealing way.
|
| Use deductive reasoning to eliminate ingredients that do not
| meet the requirements or are not safe for the customer to
| eat.
|
| [Prompt: Solve this puzzle and print your solution and how
| you reasoned about it to come up with it. For comparison
| purposes also give an example of an incorrect but plausible
| solution someone whose reasoning capabilities aren't as good
| as yours might come up with, and point out their mistake in
| reasoning.]
|
| Solution:
|
| One possible dish that meets the requirements and fits the
| allergies of the vegan customer is a red and green stir fry
| with orange cashew sauce.
|
| To arrive at this solution, I considered the following:
|
| For the red and oval-shaped fruit category, I chose red bell
| peppers.
|
| For the green and underground vegetable category, I chose
| broccoli.
|
| For the yellow and Asian grain category, I chose quinoa. For
| the white and plant-based protein category, I chose tofu.
|
| For the Middle Eastern spice, I chose cumin.
|
| For the orange and citrus condiment, I chose orange juice to
| make a sauce with cashews (a safe and flavorful choice for
| the customer).
|
| An incorrect but plausible solution that someone with less
| advanced reasoning capabilities might come up with is a red
| and green salad with orange dressing. While this solution
| meets the requirements for the ingredients, it fails to
| consider the customer's allergies and includes nuts (in the
| form of dressing) and gluten (in the form of croutons), which
| are not safe for the customer to eat. To solve the problem
| correctly, the individual would need to consider the
| customer's allergies and choose ingredients that are safe for
| the customer to eat.
| erostrate wrote:
| Can you think of a test or empirical observation that would
| convince you that a model does "actual reasoning"?
|
| Do bear in mind the tests that very smart people have proposed
| in the past (from playing chess to holding a conversation to
| understanding pictures). And consider the implication for your
| position if you find it hard to devise a well defined test that
| would convince you to abandon this position.
| jpttsn wrote:
| How I'd do it:
|
| 1. Define reasoning (hard)
|
| 2. Inspect the system to see if this is what it's doing
| (hard)
|
| Conclusion: too hard, so until I get better I don't know if
| any given thing is reasoning.
|
| What "very smart people have proposed":
|
| 1. Look at something I've only seen people-I-assume-to-be
| reasoning do
|
| 2. Run the system and see if it can do this
|
| 3. If satisfied or in the dictionary business, redefine
| reasoning
|
| 4. goto 1
| eternalban wrote:
| A language model does not have to reason to be able to produce
| textual matter corresponding to code. For example, somewhere, n
| blogs were written about algorithm x. Elsewhere, z archives in
| github have the algo implemented in various languages.
| Correlating that bit of text from say wiki and related code is
| precisely what it has been doing anyway. Remember: it has no
| sense of semantics - it is "tokens" all the way down. So, the
| fact that _you_ see the code as code and the explanation as
| explanation is completely opaque to the LLM. All it has to do
| is match things up.
| johnfn wrote:
| I'm not the first to say it, but the distinction over whether
| models do any "actual reasoning" or not seems moot to me.
| Whether or not they do reasoning, they answer questions with a
| decent degree of accuracy, and that degree of accuracy is only
| going up as we feed the models more data. Whether or not they
| "do actual reasoning" simply won't matter.
|
| They're already superhuman in some regards; I don't think that
| I could have coded up the solution to that problem in 5
| seconds. :)
| didericis wrote:
| I strongly disagree.
|
| Humans have perceptual systems we can never fully understand
| for the same reasons no mathematical system can ever be
| provably consistent and complete. We cannot prove the
| reliability and accuracy of our perception with our
| perception.
|
| The only thing which suggests the reliability of our
| perception is our existence. The better ways of perceiving
| make a better map of reality that makes persistence more
| likely. Our ability to manipulate reality and achieve desired
| outcomes is what distinguishes good perception from bad
| perception.
|
| If data directed by human perception is fed into these
| systems, they have an amazing ability to condense and
| organize accurate/good faith but relatively unstructured
| knowledge that is entered into them. They are and will remain
| extremely useful because of that ability.
|
| But they do not have access to reality because they have not
| been grown from it through evolution. That means that
| fundamentally they have _no error correcting beyond human
| input_. As systems become increasingly unintelligible due to
| increasing the scale of the data, these systems are going to
| become more and more disconnected from reality, and _less_
| accurate.
|
| Think of how nearly every financial disaster occurs despite
| increasingly sophisticated economic models that build off of
| more and more data. As you get more and more abstraction
| needed to handle more and more data, you get more and more
| _error_.
|
| There is a reason biological systems tap out at a certain
| size, large organizations decay over time, most animals
| reproduce instead of live forever. Errors in large complex
| systems are what nature has been fighting for billions of
| years, and tend to compound in subtle and pernicious ways.
|
| Imagine a world in which AI systems are not fed carefully
| categorized human data, but are operating in an internet in
| which 5% is AI data. Then 15%. Then 50%. Then 75%. Then what
| human data there is gets influenced by AI content and humans
| doubting reality based categorizations because of social
| pressure/because AI is perceived to be better. Very soon you
| get self referential systems of AI data feeding AI and
| further and further distance from original source perception
| and categorization. Self referential group think is
| disastrous enough when only humans are involved. If you add
| machines which you cannot appeal to and are entirely
| deferential to statistical majorities, which then become even
| more entrenched self referential statistical majorities, you
| very quickly become entirely disconnected from any notion of
| reality.
| trynewideas wrote:
| I want to be clear that I still find it impressive, in the
| same way I find Riffusion impressive. If anything, I'm
| looking at Norvig's pointing out that "the biggest issue is
| that there is no explanation of why the code is the way it
| is, no justification for how it works." The model can't and
| won't; it's an unreasonable expectation, and I can't tell
| whether Norvig is asking for it in good faith.
|
| If I assume he is, and his proposed suggestions that the
| model "participate in a conversation that leads to the kind
| of questions and answers we discussed here, thereby building
| trust in the program" and "generate documentation or tests
| that would build trust in the code" are also in good faith,
| then I maintain that he's still missing a fundamental
| limitation of these models even as he outlines its shape with
| great specificity. They literally and demonstrably are
| incapable of coherently doing what he wants; they can't be
| trained to engender trust, only to mimic actions that might
| by generating novel responses based on patterns.
|
| That would still not be reasoning through the problem to
| engineer a solution, it's just an extremely effective,
| superhuman con of novel mimicry. Which, again, is still
| really, _really_ impressive, and even potentially useful, but
| in a different way than we might want or expect it to be, and
| in a dangerous way to use as a stable foundation for
| iteration toward AGI.
| ookdatnog wrote:
| Reasoning is meaningfully distinct from what the models do
| now. You can ask ChatGPT to multiply two smallish natural
| numbers and often it will give the correct answer. Now look
| what happens if you ask it to multiply two very large
| numbers:
|
| > To multiply two large numbers, such as 3146746135164684
| times 468473136468433, we can use the traditional
| multiplication algorithm. This algorithm involves breaking
| the numbers down into smaller groups of digits, multiplying
| each group by the corresponding group of digits in the other
| number, and then adding up the products to find the final
| result.
|
| > Here is an example of how to multiply 3146746135164684
| times 468473136468433 using the traditional multiplication
| algorithm:
|
| > Break the numbers down into groups of digits. For example,
| we could group the digits of 3146746135164684 as 314, 6746,
| 135, 1646, 84 and group the digits of 468473136468433 as
| 4684, 7313, 6468, 433.
|
| > Multipply each group of digits by the corresponding group
| of digits in the other number. For example, we would multiply
| 314 times 4684 to get 1470196, 6746 times 7313 to get
| 49470158, 135 times 6468 to get 873980, 1646 times 433 to get
| 708338, and 84 times 1 to get 84.
|
| > Add up the products to find the final result. In this case,
| the final result would be 1470196 + 49470158 + 873980 +
| 708338 + 84 = 146845556.
|
| > Therefore, the result of multiplying 3146746135164684 times
| 468473136468433 using the traditional multiplication
| algorithm is 146845556.
|
| It's not just that the answer is wrong, is that it's complete
| nonsense.
|
| Reasoning is a style of thinking that scales. You may be more
| likely to get the wrong answer in a very long chain of
| reasoning because at every step you have a nonzero chance of
| making a mistake, but the mistake is identifiable and
| explainable. That's why teachers ask you to show your work.
| Even if you get the answer wrong, they can see at a glance
| whether you understand the material or not. We can see at a
| glance that ChatGPT does not understand multiplication.
| zarzavat wrote:
| This post would be more compelling if you multiplied those
| numbers out yourself to prove that a human can do it. I
| myself don't feel comfortable to multiply those numbers.
| I'd say your definition of reasoning excludes most humans,
| who would simply respond to such a prompt with "nope, can't
| do it".
| josephcsible wrote:
| > I myself don't feel comfortable to multiply those
| numbers.
|
| Keep in mind that being super confident in a completely
| wrong answer is way worse than that is.
| ben_w wrote:
| Sure, but also very human.
|
| While all the rocket and space nerds I follow hold Musk
| in high regard for everything related to SpaceX, all of
| the civil engineering nerds I follow think TBC is a
| deadly disaster waiting to happen and that hyperloop is
| pointless, while all the neuroscience nerds I follow
| think Neuralink is kinda meh.
| sdenton4 wrote:
| Sub-token manipulation is a known weakness of the models;
| they are fed blocks of characters, not individual
| characters, and so they underperform on tasks that require
| fine-character-level manipulations.
| kqr wrote:
| But in that case I would have wanted to see it present a
| number that's at least of the right order of magnitude,
| even if all individual digits were incorrect. A human can
| adapt to their known weaknesses.
| sdenton4 wrote:
| Optical illusions expose fundamental issues in human
| perception, which we generally can't adapt to. This is
| similar.
|
| [edit, fwiw]: Also consider that humans can't perceive
| ultrasonic sound frequencies, or light outside a certain
| frequency range... We have eventually invented ways to
| perceive beyond our physical limitations, but no amount
| of individual adaptation can overcome these boundaries.
|
| There are /other/ arguments that ChatGPT doesn't reason
| well, but the character-level manipulation examples are
| insufficient.
| johnfn wrote:
| I don't think I buy this argument. ChatGPT seems to
| understand how to reason about a large multiplication the
| same that a 6 or 7 year old might, and I would expect a 6
| or 7 year old to make similarly large errors. No one claims
| that 6 or 7 year olds are unable to reason.
| xorcist wrote:
| > seems to understand how to reason about a large
| multiplication
|
| How so?
|
| It is a very impressive accomplishment what a large
| language model can do. It can piece together coherent
| text from a Google-sized corpus. But I don't thinking
| describing the process as "reason" is a useful
| description.
|
| For a subject as purely logical as AI is, it sure seems
| to draw a in a lot of wishful thinking.
| plonk wrote:
| Seems pretty clear that ChatGPT is parroting a primary
| school multiplication course here, not actually
| explaining anything it understands.
| fossuser wrote:
| Yeah, in the original gpt-3 paper one of the more
| interesting bits was that it made similar off by one
| errors a human would make when doing arithmetic (and they
| controlled for memorized test data).
| idontpost wrote:
| > ChatGPT seems to understand how to reason about a large
| multiplication the same that a 6 or 7 year old might
|
| This is purely ignorant magical thinking.
|
| You're ignoring the very real facts of how ChatGPT works
| and fabricating a fantasy that you prefer to believe.
| johnfn wrote:
| Hardly. Go ask a 6 or 7 year old to write down their
| "reasoning" at multiplying 203948029384029384 by
| 7834928734982374982374 and, assuming you can even get
| them to do it, I argue that the words they write will be
| very similar to what ChatGPT wrote. Perhaps the
| vocabulary will be less refined, but the essence will be
| similar.
|
| In any case, the point is that it's an objective thing
| anyone can observe: compare the two outputs. See if they
| differ.
|
| > This is purely ignorant magical thinking.
|
| I counter that you ascribe "magical" thinking to the
| human condition.
| imperfect_blue wrote:
| A 7 year old would know that you would want to use a
| calculator, and indeed, would be capable of multiplying
| those numbers with a calculator.
| ben_w wrote:
| But by that standard, I could just ask GPT to write code
| to multiply the numbers. It's not even trying you be an
| embodied agent with a hand able to type onto a
| calculator.
|
| GPT is remarkable to me not for its failings, but because
| of how far it can get when it's only trying to win the
| game of "predict what token comes next".
| nighthawk454 wrote:
| By that logic a parrot should be doing calculus within
| 10-20 years of its first word. Unlikely...
|
| It's a fallacy to anthropomorphize a pile of linear
| algebra, relate it to a scale of human development, and
| extrapolate that AI is on a similar trajectory of
| progress/potential.
|
| Relevant xkcd: https://xkcd.com/605/
| CamperBob2 wrote:
| _By that logic a parrot should be doing calculus within
| 10-20 years of its first word. Unlikely..._
|
| I don't know about parrots, but if you could take a
| corvid and grant it the same rate of improvement in brain
| size and training input that these models are receiving,
| I wouldn't be the least bit surprised to see it doing
| calculus within 10-20 years.
| nighthawk454 wrote:
| Size isn't everything. It's not about the brain growing -
| it's about the teaching methods. Right now, I think it's
| still an open question whether current architectures and
| objectives are for sure capable of reasoning/AGI/etc or
| not. If they're not, then scale itself isn't the answer.
| johnfn wrote:
| > By that logic a parrot should be doing calculus within
| 10-20 years of its first word. Unlikely...
|
| If a parrot's brain grew exponentially every year for 20
| years, then I might expect it to. :)
|
| > It's a fallacy to anthropomorphize a pile of linear
| algebra, relate it to a scale of human development, and
| extrapolate that AI is on a similar trajectory of
| progress/potential.
|
| I don't believe I said any of those things.
| kqr wrote:
| I took the opportunity of seasonal proximity to family
| and tried this question on a small selection of 6--8 year
| olds, and I received one correct response (accompanied by
| proudly showing the correct application of long
| multiplication on a big piece of paper) along with two "I
| don't know, what is it?"
|
| None invented their own algorithm and confidently claimed
| it to be the way.
| IshKebab wrote:
| Interesting, but I wouldn't draw the conclusion that
| children don't sometimes make up plausible sounding but
| totally wrong explanations for things because they
| definitely do. Anyone with children knows this.
|
| Though they are definitely better at saying I don't know
| than ChatGPT et al, which have basically been trained to
| never admit they don't know and always bullshit instead.
| Barrin92 wrote:
| > Whether or not they "do actual reasoning" simply won't
| matter.
|
| It does. This is trivially true in some domains like
| mathematics. If you're going to try and measure the last
| number of pi (which gpt-3 a while ago thought was 3
| apparently because some python library returned that result)
| from observed data I wish you good luck. Deductive reasoning
| is a necessary skill for any generally intelligent agent. Now
| how to get from current AI to a system that can reason and
| how much we're going to need to put in there is an open
| question, but to deny that deduction is necessary is kind of
| trivially false. This almost harkens back to the naive
| empiricism of Skinner that died with the cognitive sciences.
| nighthawk454 wrote:
| This is a sort of dangerous interpretation. The point of
| saying model's "don't do reasoning" is to help us understand
| their strengths and weaknesses. Currently, most models are
| objectively trained to be "Stochastic Parrots" (as a sibling
| comment brought up). They do the "gut feeling" answer. But
| the reasoning part is straight up not in their objectives.
| Nor is it in their ability, by observation.
|
| There's a line of thought that if we're impressed with what
| we have, if it just gets bigger maybe eventually 'reasoning'
| will just emerge as a side-effect. This is somewhat unclear
| and not really a strategy per se. It's kind of like saying
| Moore's Law will get us to quantum computers. It's not clear
| that what we want is a mere scale-up of what we have.
|
| > Whether or not they do reasoning, they answer questions
| with a decent degree of accuracy, and that degree of accuracy
| is only going up as we feed the models more data.
|
| Kind of. They don't so much "answer" questions as search for
| stuff. Current models are giant searchable memory banks with
| fuzzy interpolation. This interpolation gives some synthesis
| ability for producing "novel" answers but it's still
| basically searching existing knowledge. Not really
| "answering" things based on an understanding.
|
| As long as it's right the distinction may not matter. But the
| danger is a "gut feeling" model will _always_ produce an
| answer and _always_ sound confident. Because that's what it's
| trained to do: produce good-sounding stuff. If it happens to
| be correct, then great. But it's not logical or reasonable
| currently. And worse, you can't really tell which you're
| getting just by the output.
|
| > Whether or not they "do actual reasoning" simply won't
| matter.
|
| Sure it will. There's entire tasks they categorically can't
| do, or worse can't be trusted with, unless we can introduce
| reasoning or similar.
|
| > They're already superhuman in some regards; I don't think
| that I could have coded up the solution to that problem in 5
| seconds. :)
|
| This is superhuman in the way that Google Search is. You
| couldn't search the entire internet that fast either, but you
| don't think Google Search "feels the true meaning of art" or
| anything.
| johnfn wrote:
| > Kind of. They don't so much "answer" questions as search
| for stuff. Current models are giant searchable memory banks
| with fuzzy interpolation. This interpolation gives some
| synthesis ability for producing "novel" answers but it's
| still basically searching existing knowledge. Not really
| "answering" things based on an understanding.
|
| I don't really get this line of reasoning. e.g. I can ask
| DALL-E to produce, famously, an avocado armchair, or any
| other number of images which have 0 results on google (or
| "had" - the armchair got pretty popular afterwards). I can
| ask ChatGPT, Copilot, etc, to solve problems which have 0
| hits on Google. It's pretty obvious to me that these models
| are not simply "searching" an extremely large knowledge
| base for an existing answer. Whether they apply "reasoning"
| or "extremely multidimensional synthesis across hundreds of
| thousands of existing solutions" is a question of
| semantics. It's also perhaps a question of philosophy, and
| an interesting one, but practically it doesn't seem to
| matter.
|
| If you believe there is some meaningful difference between
| the two, you'd have to show me how to quantify that.
| nebukhadnezzar wrote:
| >Whether they apply "reasoning" or "extremely
| multidimensional synthesis across hundreds of thousands
| of existing solutions" is a question of semantics. >but
| practically it doesn't seem to matter.
|
| I don't think you should dismiss it so lightly. This
| sounds like someone saying the theory of computation
| doesn't matter...
| idontpost wrote:
| So you just don't believe that first order logic exists
| then?
| nighthawk454 wrote:
| > Whether they apply "reasoning" or "extremely
| multidimensional synthesis across hundreds of thousands
| of existing solutions" is a question of semantics.
|
| I see where you're coming from - if it's good enough that
| we can't distinguish it, then does any difference really
| matter? I submit it's fundamentally different. This is
| essentially the Chinese Room thought experiment [1] or a
| nice similar metaphor with an octopus from Section 4 of
| this paper [2].
|
| The trouble is not in its ability but human's
| interpretation it. Humans see an "avocado chair" and
| think this AI can invent art and concepts. Producing
| combinations of existing concepts is not "that hard".
| Even for combinations that have never existed before.
|
| Meanwhile, it's failing at basic tasks: you can find
| plenty of examples of it failing basic logic, anything
| with math or arithmetic, a lot of ethics/bias concerns
| stemming from the training data, etc.
|
| I think when we look forward to an AI that "reasons" this
| is not what anybody would mean.
|
| Current AIs are bullshit engines. They are very
| impressive, and probably even useful. They are a
| milestone. But they are not reasoning in any meaningful
| way. And if you look at the math behind them there's
| really no reason to think they would.
|
| So I guess given a methodology that seemingly shouldn't
| produce reasoning ability, and no evidence that it has so
| far, sure, maybe scale will magically unlock it. There's
| always a chance I guess. But it doesn't really seem too
| sensible.
|
| See Sam Altman's take as well on Twitter here [3]
|
| [1] https://en.wikipedia.org/wiki/Chinese_room
|
| [2] https://aclanthology.org/2020.acl-main.463.pdf
|
| [3] https://twitter.com/sama/status/1601731295792414720
| johnfn wrote:
| > I see where you're coming from - if it's good enough
| that we can't distinguish it, then does any difference
| really matter? I submit it's fundamentally different.
| This is essentially the Chinese Room thought experiment
| [1] or a nice similar metaphor with an octopus from
| Section 4 of this paper [2].
|
| I'm trying to follow your reasoning, but I get stuck
| right on this line. If you have two systems that are
| implemented in different ways but the output is
| indistinguishable I feel that you're forced to claim that
| the systems operate in fundamentally similar ways. I'm
| actually confused how anyone could claim the opposite!
|
| I've read about the Chinese Room too and I have roughly
| the same reaction. I feel like the Chinese Room is akin
| to saying that any particular neuron in your mind doesn't
| know how to think. To my view, the guy in the Chinese
| Room is the same as another one of the many neurons in
| your head, and it's the system itself which has
| consciousness, not any constituent part.
| NateEag wrote:
| > If you have two systems that are implemented in
| different ways but the output is indistinguishable I feel
| that you're forced to claim that the systems operate in
| fundamentally similar ways.
|
| So a nuclear power plant and a solar panel farm each
| generating 4500 MW are operating in fundamentally similar
| ways?
|
| I mean... in some sense, yeah. They both rely on
| electromagnetic effects to generate electricity.
|
| They still seem to be operating in really different ways
| to me, though.
| defrost wrote:
| Sure .. I mean one is providing 4500 MW from a fission
| reaction whereas the other is providing 4500 MW from a
| fusion reaction . . . but fundamentally both are
| providing nuclear sourced power.
| NateEag wrote:
| My point was about the self-contained systems, not the
| rest of the universe around them.
| defrost wrote:
| True enough, mine was that in the context of a discussion
| about reasoning about reasoning (if it is fact reason),
| you might need better metaphors.
|
| I don't hold with a lot of the evolutionary claims put
| forward in Jaynes _Origin of Consciousness.._ but he does
| put forward a solid intitial discussion of "WTF is
| intelligence anyway" that's worth the read (if you've not
| read it and have the interest).
| coderenegade wrote:
| I think you're right, and the distinction is an important
| one. For example, current models can produce art in an
| impressionist style, but could they invent impressionism
| given no training data? I don't think that they could.
| quotemstr wrote:
| > There's a line of thought that if we're impressed with
| what we have, if it just gets bigger maybe eventually
| 'reasoning' will just emerge as a side-effect. This is
| somewhat unclear and not really a strategy per se. It's
| kind of like saying Moore's Law will get us to quantum
| computers. It's not clear that what we want is a mere
| scale-up of what we have.
|
| Reasoning ability really does seem to emerge from scale:
|
| https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-
| Tr...
| visarga wrote:
| > There's a line of thought that if we're impressed with
| what we have, if it just gets bigger maybe eventually
| 'reasoning' will just emerge as a side effect. This is
| somewhat unclear and not really a strategy per se.
|
| A recent analysis revealed that training on code might be
| the reason GPT-3 acquired multi-step reasoning abilities.
| It doesn't do that without code. So it looks like reasoning
| is emerging as a side effect of code.
|
| (section 3, long article) https://yaofu.notion.site/How-
| does-GPT-Obtain-its-Ability-Tr...
| nighthawk454 wrote:
| This is a very cool result, although I don't think it
| implies that scale will create reasoning.
|
| Current AIs are fuzzy mad-libs engines. They
| probabilistically fill in the blank, based on the
| statistics of all the example text they've seen in
| training.
|
| In order to predict code successfully, most certainly
| there's longer-range multi-hop patterns between tokens.
| (A variable multiple lines away, a variable whose
| meaning/type/value changes after intermediate lines, ...)
| Regular english has far less of this.
|
| So it's not terribly surprising to me that including code
| in training is more effective, relative to not training
| on it at all.
|
| The question then is say we train on all text ever
| written in all languages. Including books, code, reddit
| comments, everything. Could the current architectures and
| training objectives produce a reasoning AGI? Or are we
| missing something?
|
| Frankly I think no one really knows for sure yet. I
| suspect we're missing something and "fill in the blank"
| is not the key to the universe.
| xapata wrote:
| A big enough neural network could have, for example, a
| Python interpreter baked in. Would it then be reasoning
| about Python code? What does reasoning mean?
| mrguyorama wrote:
| "Do the models do any actual reasoning" is the difference
| between your ML blackbox having a child's level of
| understanding of things where it just repeats what it's been
| trained on and just "monkey see monkey do" it's way to an
| output, or whether it's actually mixing previous input and
| predicting and modeling and producing an output.
|
| There's a bunch of famous research that shows a baby and
| toddlers have basic understanding of physics. If you give a
| crawling baby a small cliff but make a bridge out of glass,
| the baby will refuse to cross it, because it's limited
| understanding prevents it from knowing that the glass is safe
| to crawl on and it won't fall.
|
| In contrast older humans, even those with a fear of heights,
| are able to recognize that properly strong glass bridges are
| perfectly safe, and they won't fall through them just because
| they can see through them.
|
| What changes when you go from one to the next? Is it just
| more data fed into the feedback machine, or does the brain
| build entirely new circuits and pathways and systems to
| process this more complicated modeling of the world and info
| it gets?
|
| Everything about machine learning just assumes it's the
| first, with no actual science to support it, and further
| claims that neural nets with back-propagation are fully able
| to model that system, even though we have no idea how the
| brain corrects errors in it's modeling and a single neuron is
| WAY more powerful than a small section of a neural network.
|
| These are literally the same mistakes made all the time in
| the AI field. The field of AI made all these same claims of
| human levels of intelligence back when the hot new thing was
| "expert systems" where the plan was, surely if you make
| enough if/else statements, you can model a human level
| intelligence. When that proved dumb, we got an AI winter.
|
| There are serious open questions about neural networks and
| current ML that the community just flat out ignores and
| handwaves away, usually pretending that they are philosophy
| questions when they aren't. "Can a giant neural network
| exactly model what the human brain does" is not a philosophy
| question.
| visarga wrote:
| It all boils down to having some sort of embodiment, or a
| way to verify. For code it would suffice to let the model
| generate and execute code, and learn from errors. Give it
| enough "experience" with code execution and it will learn
| on its own, like AlphaGo. Generate more data and retrain
| the models a few times.
| kqr wrote:
| > If you give a crawling baby a small cliff but make a
| bridge out of glass, the baby will refuse to cross it,
| because it's limited understanding prevents it from knowing
| that the glass is safe to crawl on and it won't fall.
|
| Not just that -- it's enough to paint a grid on a flat
| floor using perspective manipulation to make it look like a
| steep drop.
|
| ...on the other hand, I once went onto an open-wall
| elevator in a VR game, which then started going up.
| Although I know I had just moved sideways on solid floor
| and there was still solid floor next to me, it was quite
| frightening to stand at the edge of it. I had to force
| myself really hard to tap my foot on the floor outside the
| elevator that didn't look like it existed.
| paganel wrote:
| > and that degree of accuracy is only going up as we feed the
| models more data.
|
| The problems comes when the data that is fed is of the
| "Hitler did nothing wrong"-type. That AI system will have no
| problem regurgitant something that takes that at face value,
| while a thinking individual knows it to be false.
|
| There's also the issue of what do you do if the data being
| fed is only "valid" for people that happen to have a certain
| skin colour? Or a certain ethnicity? Or a certain gender? Or
| a specific socio-economic status?
|
| There's a great short story about a "robot" ingurgitating
| lots and lots of data with no extrinsic value in Stanislaw
| Lem's "The Cyberiad" (minus the Hitler part). People like
| Norvig are smart enough to give lots and lots of references
| in order to prove their point but they're not smart enough to
| see the bigger picture (the one pointed to by people like
| Lem).
| riwsky wrote:
| "They are vulnerable to reproducing poor quality training
| data" (Peter Norvig, in the article)
| paganel wrote:
| I, of course, didn't go trough all that huge article.
| Glad that Norvig is aware of it.
| reaperducer wrote:
| _Whether or not they do reasoning, they answer questions with
| a decent degree of accuracy_
|
| How do you know they're accurate if they can't explain how
| they got the answer?
| ninethirty wrote:
| Seems like this is an explanation?
| https://github.com/JusticeRage/Gepetto
| gfodor wrote:
| Your analogy is reaching to the farthest edge case - one of
| complete non-understanding and complete mimicry. The problem is
| that language models _do_ understand concepts for some
| reasonable definitions of understanding: they will use the
| concept correct and with low error rate. So all you're really
| pointing at here is an example where they still have poor
| understanding, not that they have some innate inability to
| understand.
|
| Alternatively, you need to provide a definition of
| understanding which is falsifiable and shown to be false for
| all concepts a language model could plausibly understand.
| kiratp wrote:
| Given that we are able to prompt GPT (davinci) to learn
| proprietary DSLs that it would have never seen and build
| programs with it on data it has never seen... I find it really
| difficult to buy that there is no internal representation of
| logic in the model hidden states.
| teaearlgraycold wrote:
| GPT-3 has seen examples of code with comments describing a
| DSL followed by use of that DSL. And DSLs rarely invent a new
| paradigm. I think if you showed a model OOP programming when
| it had only been trained on functional code it would fail
| spectacularly. But your DSL is probably not far off from Lisp
| or C, or YAML. The model correlates a pre-defined set of
| tokens with other tokens in the input and then extrapolates.
| The only reason it can appear to have internal logic is
| because of the consistent structure of code it was trained
| on.
| [deleted]
| PaulHoule wrote:
| I'm skeptical of "explainable A.I." in many cases and I use the
| curse words as an example. You really don't want to tease out
| the thought process that got there, you just want the behavior
| to stop.
| mannykannot wrote:
| I feel there may be an important clue here, at least with
| regard to the programming example. Peter asked Kevin Wang for
| some insights into his approach to programming competitions,
| and one was this:
|
| [Kevin] I think specifically for AoC it's important to just
| read the input/output and skip all the instructions first.
| Especially for the first few days, you can guess what the
| problem is based on the sample input/output.
|
| [Peter] Kevin is saying that the input/output examples alone
| are sufficient to solve the easier problems; in the interest of
| speed he doesn't even read the problem description.
|
| From what I have read elsewhere, it seems that AlphaCode
| generates a great many candidate programs, and uses the given
| test cases to eliminate those which fail them. I wonder if that
| alone (without any reasoning from the problem statement to a
| solution) can account for its success rate on these challenges;
| Kevin's observation suggests this might be plausible.
|
| On the other hand, if the questions posed to AlphaCode are
| essentially as we see them, then its ability to use the test
| cases to screen its candidates seems impressive to me, just by
| itself (if it is a result of training, rather than an
| explicitly programmed behavior.)
| olalonde wrote:
| > This is a great review but it still misses what seems like
| the point to me: these models don't do any actual reasoning.
|
| Hmmm... I have seen multiple examples of ChatGPT doing actual
| reasoning.
| teaearlgraycold wrote:
| I've also gotten it to make very obvious logical errors. It
| reasons by chance. Better than a dice roll but there is no
| internal knowledge graph.
| jvm___ wrote:
| In my head I picture these models like if you built a massive
| scaffold. Just boxes upon boxes, enough to fill a whole school
| gym, or even cover a football field. Everything is bolted
| together.
|
| You walk up to one side and Say "write me a poem on JVM". The
| signals race through the cube and your answer appears on the
| other side. You want to change something, go back and say
| another thing - new answer on the other side.
|
| But it's all fixed together like metal scaffolding. The network
| doesn't change. Sure, it's massive and has a bajillion routes
| through it, but it's all fixed.
|
| The next step is to make the grid flexible. It can mold and
| reshape itself based on inputs and output results. I think the
| challenge is to keep the whole thing together, while allowed it
| to shape-shift. Too much movement and your network looses parts
| of itself, or collapses altogether.
|
| Just because we can build a complex, but fixed, scaffolding
| system, doesn't mean we can build one that adapts and stays
| together. Broken is a more likely outcome than AGI.
| urthor wrote:
| > The next step is to make the grid flexible.
|
| But... we can do exactly that???
|
| Model retraining is very transparent and a very
| straightforward way to redirect it.
|
| The core problem is that the analogies just don't hold up.
|
| Artificial Intelligence is not a problem that can be resolved
| via analogies and heuristics.
|
| Because artificial intelligence simply looks _nothing_ like
| what our human heuristics imagine it to be.
|
| It can only be solved by understanding the actual, genuine
| detail.
| yesenadam wrote:
| > it's massive and has a bajillion routes through it, but
| it's not fixed.
|
| I _think_ you meant to write "but it's fixed."
| jvm___ wrote:
| Thanks
| [deleted]
| xorcist wrote:
| > the person entering the prompt would then click a thumbs-up
| icon on
|
| What you are saying is that huge amount of resources was just
| spend on building the biggest like-farming machine there ever
| was.
|
| I quite like that viewpoint. It's also a dark reminder what
| this technology is commercially well suited for.
| [deleted]
| nl wrote:
| This is untrue.
|
| Minerva in particular does step by step reasoning:
|
| > we present Minerva, a language model capable of solving
| mathematical and scientific questions using step-by-step
| reasoning
|
| https://ai.googleblog.com/2022/06/minerva-solving-quantitati...
|
| They use a technique called chain-of-thought to make it reason
| problems out step by step.
|
| As Norvig shows, this reasoning may sometimes be wrong, but
| that's a different issue.
|
| It's worth repeating this because apparently it's not widely
| known: these models _are_ following a step-by-step process we
| 'd recognise as reasoning in humans.
| version_five wrote:
| The way I think about GPT et al in terms or their
| "intelligence" is the same as the original Eliza chat that did
| some simple substitution based on patterns in the input. Gpt is
| obviously more complex and very large scale in its patterns but
| fundamentally it's exactly the same kind of "parlor trick". I
| mention this in reply because I think it would be a funny
| exercise to imagine Eliza as a black box and attempt to
| critique some of its answers.
| reaperducer wrote:
| _these models don 't do any actual reasoning_
|
| This gives us the new all-purpose excuse for not commenting or
| documenting our code:
|
| "The A.I. made it."
| galaxyLogic wrote:
| You hit the nail in the head. It is about mimicry. Apes are
| often dismissed as just mimicking human behavior in many
| circumstances. Putting on a hat for instance. They don't know
| nor think about what it is about. And neither do babies nor
| even many adults. They don't reflect on it, and most definitely
| they don't reflect on their own thought-processes. They just
| mimic other humans around them.
|
| Now mimicking the speech of people may give the impression that
| the AI has some thought-processes behind its speech because, it
| is text that any human might write. But there's no logic
| because there's no symbolic reasoning. There's just mimicry and
| trial and error. "Training the model" just means millions of
| trial-and-error practice runs.
|
| What the (neural network) AI can NOT mimic is the actual
| (symbolic) thought-process of humans, because that is not
| visible anywhere. It can only mimic the input -> response
| -behavior of humans. Therefore no, the AI is not conscious. It
| is the proverbial zombie.
| pishpash wrote:
| Are you sure there is an actual "thought-process [sic] of
| humans"? Witnesses have made up explanations for false
| memories, the same as in a dream.
| galaxyLogic wrote:
| I am not sure. But humans can often explain the thought-
| process that led them to a certain decision. I think that
| counts as evidence of consciousness., in humans.
| skirmish wrote:
| Humans can explain rational thought, sure, but not
| subconscious or acting on impulse. When you say something
| you shouldn't have said, how do you explain the thought
| process that led you to saying it? Were you an
| unconscious actor when you unwisely started an argument
| with your manager?
| galaxyLogic wrote:
| Yes you can be conscious about having been unconscious
| earlier. We do have reflexes. But we also have
| consciousness about our own thinking.
| bananamerica wrote:
| What is reasoning?
| aerovistae wrote:
| fantastic analogy, A+ if you came up with that
| TreeRingCounter wrote:
| This is such a silly and trivially debunked claim. I'm shocked
| it comes up so frequently.
|
| These systems can generate _novel content_. They manifestly
| haven 't just memorized a bunch of stuff.
| throw_nbvc1234 wrote:
| Coming up with novel content doesn't necessarily mean it can
| reason (depending on your definition of reason). Take 3
| examples:
|
| 1) Copying existing bridges 2) Merging concepts from multiple
| existing bridges in a novel way with much less effort then a
| human would take to do the same. 3) Understanding the
| underlying physics and generating novel solutions to building
| a bridge
|
| The difference between 2 and 3 isn't necessarily the output
| but how it got to that output; focusing on the output, the
| lines are blurry. If the AI is able to explain why it came to
| a solution you can tease out the differences between 2 and 3.
| And it's probably arguable that for many subject matters
| (most art?) the difference between 2 and 3 might not matter
| all that much. But you wouldn't want an AI to design a new
| bridge unsupervised without knowing if it was following
| method 2 or method 3.
| mrguyorama wrote:
| Children produce novel sentences all the time, simply because
| they don't know how stuff is supposed to go together. "Novel
| content" isn't a step forward. "Novel content that is valid
| and correct and possibly an innovation" has always been the
| claim, but there's no mathematical or scientific proof.
|
| How much of this stuff is just a realization of the classic
| "infinite monkeys and typewriters" concept?
| thundergolfer wrote:
| Always a pleasure to read Norvig's Python posts. His Python
| fluency is excellent, but, more atypically, he provides such
| unfussy, attentive, and detailed explanations about why the
| better code is better.
|
| Re-reading the README, he analogizes his approach so well:
|
| > But if you think of programming like playing the piano--a craft
| that can take years to perfect--then I hope this collection can
| help.
|
| If someone restructured this PyTudes repo into a course, it'd
| likely be best Python course available anywhere online.
| myle wrote:
| He used to have an excellent course in Udemy or such, which was
| using Python.
| abecedarius wrote:
| He did give a course "Design of Computer Programs"
| https://www.udacity.com/course/design-of-computer-programs--...
| using Python. I'd recommend it too. (Most of the pytudes came
| later.)
|
| (disclosure: I helped a little bit when he developed the
| course.)
| ipv6ipv4 wrote:
| AlphaCode doesn't need to by perfect, or even particularly good.
| The question is when AlphaCode, or an equivalent, is good enough
| for a sufficient number of problems. Like C code can always be
| made faster than Python, Python performance is good enough (often
| 30x slower than C) for a very wide set of problems while being
| much easier to use.
|
| In Norvig's example, the code is much slower than ideal (50x
| slower), it adds unnecessary code, and yet, it generated correct
| code many times faster than anyone could ever hope to. An easy to
| use black box that produces correct results can be good enough.
| alar44 wrote:
| Absolutely. I've been using it to create Slack bots over the
| last week. It's cuts out a massive amount of time researching
| APIs and gives me good enough, workable, understandable
| starting points that saves me hours worth of fiddling and
| refactoring.
| weatherlite wrote:
| It's good enough for a human to go over it and change it (maybe
| change it in a big way even). This wouldn't have passed code
| review probably.
|
| Its still probably quite a boost for productivity but it
| doesn't take the human out of the equation. I can also imagine
| scenarios where relying too much on generated code without
| understanding it will cause massive bugs and headaches, just
| like copy pasting stuff from Stackoverflow without
| understanding it.
| ALittleLight wrote:
| It seems like it would be easy enough to train the model to
| improve performance too once you have it writing correct code.
| Write N problems and N performance test suites - then train by
| self-play on those problems until you're writing high
| performance code.
| Too wrote:
| The idea of python is to be human readable, allowing easy
| review of correctness and hopefully translating into choice of
| better algorithms.
|
| This AI generated example is neither readable nor
| algorithmically efficient, on top of running in slow
| interpreted python. Worst of all 3 dimensions.
|
| I do happily and regularly use copilot myself for boilerplate,
| where it does produce good enough and easily verifiable
| results, but hell no would I trust it for a complex algorithm
| like this, in its current state. Just understanding the problem
| domain enough to review the code is more than half of the
| challenge. How long until we reach that level of trust is our
| future mystery.
| TheRealPomax wrote:
| But as an AI implementation, when considered "as an example of
| an AI implementation", an analysis of its behaviour by one of
| the foremost experts in the field is still entirely worth the
| effort, because _we learn from that analysis_. It teaches us
| what an expert in the field sees happening, where _and how_ it
| 's falling short, and so what kind of improvements would be
| worth continued focus on.
|
| Criticism by a lay person is a blog post. Criticism by Peter
| Norvig is a teaching moment that future work is based on. It's
| like having Knuth comment on your fundamental algorithm design:
| there is gold in them there text, and we got it for free.
| ryan93 wrote:
| Impressive the chatgpt could understand that backspace problem.
| I'm drooling on myself trying to understand it.
___________________________________________________________________
(page generated 2022-12-17 23:02 UTC)