[HN Gopher] The continued relevance of Weizenbaum's warnings
___________________________________________________________________
The continued relevance of Weizenbaum's warnings
Author : pabs3
Score : 109 points
Date : 2023-01-27 07:20 UTC (15 hours ago)
(HTM) web link (librarianshipwreck.wordpress.com)
(TXT) w3m dump (librarianshipwreck.wordpress.com)
| Radim wrote:
| Article so wordy it could use an AI summary, but this quote
| stands out:
|
| > _Writing of the enthusiastic embrace of a fully computerized
| world, Weizenbaum grumbled, "These people see the technical
| apparatus underlying Orwell's 1984 and, like children on seeing
| the beach, they run for it"_
|
| > _a point to which Weizenbaum added "I wish it were their
| private excursion, but they demand that we all come along."_
| lxgr wrote:
| In my computer science undergrad program, we watched and
| discussed a movie about Joseph Weizenbaum and Ray Kurzweil. I
| still vividly remember that discussion.
|
| At the time, it seemed obvious to me that Kurzweil got it right
| and Weizenbaum got it wrong; that clearly, ELIZA was a red
| herring; that people wouldn't be fooled that easily; that the
| first AI to truly pass the Turing test would be the real thing
| (and we'd definitely see one, and with that the Singularity, in
| my lifetime).
|
| I think about Weizenbaum a lot these days.
| throwanem wrote:
| Was it _Plug & Pray_? If so, I'll add it to my list.
| lxgr wrote:
| Not 100% sure, but that sounds like it.
| agentultra wrote:
| I get the sense that when people _want_ to believe something is
| true they will willingly ignore evidence to the contrary. They
| want to believe that consciousness can form from a giant
| spreadsheet of inferences. And so they will find evidence to
| support that theory.
|
| _ChatGPT will change the world!_ Sure, it already has. If we pay
| attention to the outcomes and see evidence of how it 's changing
| the world.. it's mostly people doing the changing with their
| desires, beliefs, and politics. Same as always. The system itself
| is nothing but an automaton.
|
| _But aren 't we all automatons too, building our self awareness
| around models of the word?_ Ah the AI version of the, "You can't
| prove that it's not consciousness!" Well we can as many
| scientists studying consciousness in humans and animals can
| attest: it's complicated, but we're getting better at
| understanding what it is. ChatGPT ain't it.
|
| Definitely worth looking at the motivations of the people
| building the technology itself.
| lukeschlather wrote:
| I feel like most of the discussion of this topic is muddled by
| conflation of a few different questions. One is whether you want
| human or procedural decisionmaking to guide a task. The second
| question is what distinguishes human and procedural
| decisionmaking. The third is whether or not computers are capable
| of executing a given procedure.
|
| As for "distinguishing human and procedural decisionmaking" I
| would define it as basically "human" decisionmaking is when there
| is some procedure but the human will disregard procedure based on
| some model of the "intent." You have to trust the human for this
| to work. So there's a fourth question, which is under what
| circumstances you could trust a computer to correctly model your
| intent. But then there's a fifth question, which is "whose
| intent?" But whose intentions the process serves isn't really
| intrinsically tied to whether humans or computers are running the
| procedure.
|
| Another thing is that Weizenbaum and the author seem to take it
| for granted that human decisionmaking is literally magic and a
| computer is incapable of that kind of magic. But it's unclear and
| I think it's an important thing to state outright.
| mjburgess wrote:
| It's not just intention. It's, as some times dismissively put,
| "common sense".
|
| What price is an airline ticket on the first day of a global
| pandemic? No one knows, and there isn't anything to provide a
| "procedural system" in order for it to model what our
| "intention" would be in those cases.
|
| This seems obvious with a pandemic, but it is common place.
| Every day is a genuinely novel occurence, and counts only as a
| repetition of some superficial patterns in very very limited
| respects.
|
| When this novelty becomes _relevant_ to a problem can never,
| (obviously), be pre-stated because we don 't yet know what that
| novelty is.
|
| There is simply no substitute for being-in-the-world as animals
| are, that is: moving around, communicating, acting, reacting
| and _participating_. Growing, learning, reproducing, and
| problem-solving in _direct causal contact with reality_.
|
| This isn't magic, but there is no "procedural model" of it; nor
| should we expect there to be. There aren't procedural models of
| almost any phenomenon.
|
| What _we_ can _build_ a computer to do is "almost nothing",
| and we shouldnt expect the trivial problem space given by this
| to exhaust what we actually _need_ machines to do.
| pixl97 wrote:
| I mean it seems there is some conflation between the human
| mind and human senses, and the computer mind and computer
| sensing going on here.
|
| You're stating that a large part of human behavior is
| reacting to the environment presented to them via sensory
| input/output mechanisms. What you have not stated is why a
| machine entity cannot do the same things?
| throwanem wrote:
| > The second question is what distinguishes human and
| procedural decisionmaking.
|
| Nothing, because any procedure that exists was necessarily
| defined by humans. If the procedure is implemented by a
| computer, at most that provides humans an excuse for choosing
| not to disregard an outcome it produces.
|
| > Another thing is that Weizenbaum and the author seem to take
| it for granted that human decisionmaking is literally magic
|
| I can't imagine where this idea is coming from. The thrust of
| the argument is not that human decisions can't be modeled, but
| that omitting the question of whether they _should_ be modeled,
| and those models granted (what is necessarily the pretense of)
| authority to make decisions in human stead, evades the same
| sort of responsibility that a civil engineer has to ensure that
| his bridge design can withstand the mechanical loads that it
| will bear in use.
| throwanem wrote:
| (Past the edit window, but belatedly to add: This is why
| "code is law" and "law is code" are both nonsense, indeed the
| _same_ nonsense approached from varying directions.)
| qsort wrote:
| Also the question of whether or not a technology is neutral is
| separate from the question of whether or not a particular
| implementation of that technology is.
|
| Which is itself separate from the question of whether or not
| Mark Zuckerberg is being kind of a dick.
|
| The article isn't bad and the reference is worth reading, but
| it's impossible to engage with this much material at the same
| time as though it's one conversation. It's poor intellectual
| discipline, I'm sorry.
| labrador wrote:
| The first time I sensed I was dealing with an alien intelligence
| was playing a chess machine in 1980. I remember being perplexed
| that it beat me and aware the machine embodied intelligence in
| some way that I didn't understand
|
| It's not really clear to me if this article is directed at
| scientists and engineers getting carried away or the general
| public. If the former, yes they should know better, but the
| general public is mostly not rational, contrary to what I used to
| think about the reasoning person. Many people attribute thoughts
| and feeling to their pets that they don't have. They think
| crystals have special powers. It's only natural that they will
| think AI have consciousness. I think as scientists and engineers
| we have a responsibility to prevent our AI creations from telling
| people to drink the poisoned Kool Aid or eat glass. Yann LeCun
| said they shut down Meta's chat AI because it told people that
| suicide had advantages and one can eat glass. Google is similarly
| cautious. OpenAI came along and let the cat out of the bag. I
| wonder if Microsoft has thought through what might happen if they
| hook it to Bing and it starts telling people crazy things. All it
| took was a couple of posts on 4chan to get the Q Anon movement
| off the ground. anthropomorphize
| attribute human characteristics or behavior to (a god, animal, or
| object). "people's tendency to anthropomorphize their
| dogs" numinous having a strong religious or
| spiritual quality; indicating or suggesting the presence of a
| divinity. "the strange, numinous beauty of this ancient
| landmark" What is an example of numinous?
| Something numinous has a strong religious quality, suggesting the
| presence of a divine power. When you enter a temple, church, or
| mosque, you might feel as though you've entered a numinous space.
|
| Edit: Write me today's astrology for a Leo
|
| ChatGPT: I'm sorry, I am not able to provide astrology
| predictions as it is not based on scientific evidence. It is a
| form of divination that is not considered reliable. Is there
| anything else I can help you with?
| noorkersz wrote:
| > _to prevent our AI creations from telling people to drink the
| poisoned Kool Aid or eat glass_
|
| yea, let's leave this job to evil and ignorant humans...
| danbruc wrote:
| _It 's only natural that they will think AI have
| consciousness._
|
| That seems just a matter of time until they would be doing this
| rightly so.
| labrador wrote:
| You have to explain consciousness in humans before you can
| tell me it has arisen in silicon chips
| int_19h wrote:
| This cuts both ways - you have to explain consciousness in
| humans before telling that the chips are definitely not
| conscious, no?
| birracerveza wrote:
| Exactly. Can you explain it humans? No, yet you assume
| they're conscious. In animals? No, yet you assume they're
| conscious, while acknowledging their "consciousness" is
| vastly different than humans. Same reasoning applies with
| AIs.
| labrador wrote:
| I think therefore I am. I am aware of being conscious and
| thinking thoughts. I don't know if others are conscious.
| Maybe others are robots (Kurt Vonnegut - Breakfast of
| Champions). Maybe others are NPCs (non-player
| characters). This is the modern way of calling people
| robots. Kurt Vonnegut's solution was to find someone who
| has a creative spirit, an artist that has a light within
| them (soul.)
| danbruc wrote:
| My personal guess is that we will first ask a computer if
| it is conscious and it will truthfully answer yes. Just in
| the same way you would try to figure out if another fellow
| human is conscious and in the same way a human would
| respond.
|
| Then some time later as we figure out how consciousness
| works in the computer - as it is much easier to inspect
| than a human - and compare the findings with humans to see
| if they indeed work in a similar way, we will all be a bit
| disappointed that the solution is much more mundane than we
| thought or hoped.
| whatshisface wrote:
| The problem with your scheme is that GPT is already able
| to answer yes and the only reason Chat GPT doesn't is
| that it has been deliberately bent by OpenAI not to do
| it. In the absence of reinforcement learning, large
| language models spit out text according to how they think
| text normally goes. It's a text generation procedure that
| has nothing to do with beliefs or inner thoughts - all
| the model knows about is what word comes next.
| danbruc wrote:
| _It 's a text generation procedure that has nothing to do
| with beliefs or inner thoughts - all the model knows
| about is what word comes next._
|
| How do you know that? To be clear, I totally agree that
| ChatGPT is not yet there and I also think that scaling it
| up will not get us there, but I think the gap might be
| much smaller than most people think.
|
| The way we think is often divided into two categories,
| for simple things like 1 + 1 you just know the answer,
| for complicated things things like 13 * 47 we really have
| to think and reason in steps. ChatGPT seems to do pretty
| well in the first category but it is not really capable
| of doing things in the second category. On the other hand
| I have seen examples of people talking ChatGPT through a
| reasoning process to arrive at the correct answer for
| something that it got initially wrong, for example ROT13
| encoding some text.
|
| So what if we stuff two copies of ChatGPT into a black
| box and instead of just spitting out what ever ChatGPT
| spits out, we let the two copies first have some inner
| dialog? I don't think it is perfectly obvious how one
| would do this or what the result would be, but I think
| ChatGPT has enough basic knowledge that there is at least
| a chance that one could get it to reason in a step by
| step fashion.
| burnished wrote:
| Do you know how these models work? I mean no offense but
| it seems like you do not, and your imagination is filling
| in the gaps.
| noorkersz wrote:
| you expect a written explanation of an incomplete ongoing
| process
|
| but 'written explanations' only really work for done and
| completed things, or sometimes for future things which we
| want to repeat; i.e. which are based on things we've done
| before
| lamacase wrote:
| I don't think that's a fair bar. There are arguments for
| consciousness in non-human animals that don't depend on a
| full explanation of consciousness in humans.
|
| But that also doesn't imply we should suspect any
| particular AI of being conscious just because AI in general
| could be conscious in principle. We don't expect worms to
| be conscious just because animals can be.
| labrador wrote:
| I wasn't clear. I'm talking about consciousness in
| general, humans, animals, octopi... I keep reading but
| haven't found an explanation that satisfies me. Ray
| Kurzweil has some interesting things to say about
| computers and consciousness in his "The Age of Spiritual
| Machines". I was skeptical about a lot of what he wrote
| when I read it many years ago.
|
| Now I'm not sure it matters. If people imbue their
| devices with intelligence and emotions then that's what
| they perceive. I've since learned about Tulpas. Some
| people have imaginary friends that seem very real to
| them. People will likely move their Tulpas out of their
| minds and into their AI's.
|
| https://en.wikipedia.org/wiki/Tulpa
|
| People on reddit are already talking about Tulpa's and AI
|
| https://old.reddit.com/r/Tulpa/
| hexciple wrote:
| Why, though? Wouldn't the logical next step of "we don't
| understand what human consciousness is" be that we can't
| determine if it's happened in another being, rather than
| it's not possible for it to have happened there?
| burnished wrote:
| You agree with them. 'You cant say it has risen' is
| equivalent to 'we cant determine if' for purposes of this
| discussion.
| Zigurd wrote:
| This is the part I have to disagree with. If AGI turns out
| to be emergent and depends on scaling up simple mechanisms,
| that emergence is all the "explain consciousness" you need.
| Dogs, corvids, octopods, bononos, etc. do not have human
| intelligence but they all have brains that perform some
| tasks our brains perform. That suggests that a combination
| of scale, a particular set of architectural parameters
| within the range of possibilities of mammals, and, maybe,
| some unique variation is what it takes to make a human
| intelligence. Nobody explained consciousness before we
| labelled this thing "consciousness."
| burnished wrote:
| >Nobody explained consciousness before we labelled this
| thing "consciousness."
|
| I think that is a core part of the difficulty coming up
| in these discussions.
| adamsmith143 wrote:
| "You have to solve the navier stokes equation before you
| can say an airplane is truly flying."
|
| I hope you realize how silly that sounds.
| labrador wrote:
| I can tell when an airplane is flying and I know how
| machines are able to fly. I have no way of knowing if you
| are conscious or how you are conscious if you say you
| are.
| adamsmith143 wrote:
| >I wonder if Microsoft has thought through what might happen if
| they hook it to Bing and it starts telling people crazy things.
| All it took was a couple of posts on 4chan to get the Q Anon
| movement off the ground.
|
| https://www.perplexity.ai/ already did this and its really good
| and not this dystopian thing you luddites keep imagining.
| labrador wrote:
| I am neither an Luddite or pollyanna about AI. I'm an
| engineer looking for failure modes.
| melagonster wrote:
| you immediately started to attack him, why ...
| rhelz wrote:
| C.F. the "AI effect": once you understand how the "magic" is done
| by very simple individual steps it stops being magical and you
| tend to think "that's not AI, its just computation." After
| thinking this for a while, you conclude that AI hasn't gotten
| anywhere despite decades of effort.
|
| Perhaps we are finally thinking that AI is getting somewhere
| precisely because we don't understand how neural nets really work
| yet. It keeps the magic alive long enough for us to realize that
| progress really is being made.
| sumtechguy wrote:
| The first thing I can think of that was like the 'AI' we have
| now is seeing the cubic spline method. It was shockingly good
| at particular points. But between those points it could be 'meh
| close enough' or wildly bad. AI we have now seems more along
| the lines of 'meh close enough'. Which for many things is
| actually good enough. For me I have always looked at the nets
| as a cubic spline smeared across dozens of nodes with a newtons
| method for searching out 'optimal'. Bad analogy I know. But
| that is the closest I can get to explaining 'how they work'.
| int_19h wrote:
| The other factor here is that we appear to be trying really
| hard to keep it "magic" wrt ourselves. At the very least, the
| notion that it might be "just computation" in the brain seems
| to be very discomforting to some people.
|
| Which might affect the way we look at technologies that try to
| model it, or at least use similar approaches - i.e. we're
| primed to treat them as more "magic" than they really are, or
| else to invent some reason why we think ourselves to be
| different.
| walleeee wrote:
| > technology does not drive history, people drive history
|
| this fails to grasp the changing nature of agency in a world
| brimming with hybrid intelligence
|
| we would perhaps like to think we and we alone "drive" history,
| and I agree with the author that we _ought_ to try to, but
| technology carries its own inertia
|
| nonhuman agency accretes as we interact with our implements and
| artifacts
|
| this occurs even before we explicitly design autonomy into our
| tools, which of course we are now doing
|
| technology carves ruts in the mind and in the world
| grantcas wrote:
| It's becoming clear that with all the brain and consciousness
| theories out there, the proof will be in the pudding. By this I
| mean, can any particular theory be used to create a human adult
| level conscious machine. My bet is on the late Gerald Edelman's
| Extended Theory of Neuronal Group Selection. The lead group in
| robotics based on this theory is the Neurorobotics Lab at UC at
| Irvine. Dr. Edelman distinguished between primary consciousness,
| which came first in evolution, and that humans share with other
| conscious animals, and higher order consciousness, which came to
| only humans with the acquisition of language. A machine with
| primary consciousness will probably have to come first.
|
| What I find special about the TNGS is the Darwin series of
| automata created at the Neurosciences Institute by Dr. Edelman
| and his colleagues in the 1990's and 2000's. These machines
| perform in the real world, not in a restricted simulated world,
| and display convincing physical behavior indicative of higher
| psychological functions necessary for consciousness, such as
| perceptual categorization, memory, and learning. They are based
| on realistic models of the parts of the biological brain that the
| theory claims subserve these functions. The extended TNGS allows
| for the emergence of consciousness based only on further
| evolutionary development of the brain areas responsible for these
| functions, in a parsimonious way. No other research I've
| encountered is anywhere near as convincing.
|
| I post because on almost every video and article about the brain
| and consciousness that I encounter, the attitude seems to be that
| we still know next to nothing about how the brain and
| consciousness work; that there's lots of data but no unifying
| theory. I believe the extended TNGS is that theory. My motivation
| is to keep that theory in front of the public. And obviously, I
| consider it the route to a truly conscious machine, primary and
| higher-order.
|
| My advice to people who want to create a conscious machine is to
| seriously ground themselves in the extended TNGS and the Darwin
| automata first, and proceed from there, by applying to Jeff
| Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a
| conscious machine is at https://arxiv.org/abs/2105.10461
| throwanem wrote:
| This is wildly off topic for a thread from an article that
| touches nowhere near the question of whether a machine can be
| made conscious.
| cmrdporcupine wrote:
| This is a great essay. I'm going to read it a few times.
|
| One thing I have thought about over the years more and more, in
| this general vein is that it is sometimes to think about
| "computing technology" as being an evolution less as a specific
| thing itself and more of a broader "automation technology" which
| emerges most dramatically with the arrival of the industrial
| revolution (though there were aspects of it before that.)
|
| Thinking with a kind of materialist analysis, we can see
| computing but even (more broadly) higher level mathematics,
| logic, etc. as an intellectual effort to chip away at labour
| costs & the percentage of human effort involved in
| production/work/business. It's a deeply obscured profit motive.
|
| The ideological blindness described in this article I think fits
| neatly into that: we are all so enamoured with this computing
| thing, this process, the puzzle and the end results that we often
| fail to see that it is not some magical force operating under its
| own motive, with its own natural inevitable destination and we
| often do not see that there's likely a deeper force underlying
| all of this... human efforts, labour, driven by profit and
| hunger.
|
| Ultimately computing technology is capitalist automation
| technology. It reflects the needs of the capitalist market
| system, and the drive to efficiency that the market economy
| requires. The 'magic' we seem to pepper overtop is our own
| flourish.
|
| (Maybe when I'm finally done with this software engineering
| career that has eaten my life, I will go back and finish my
| philosophy degree I walked away from in the 90s... and then I
| will write more coherently about this)
| CatWChainsaw wrote:
| We need more people talking about this seriously now, because
| we are on a precipice of some sort.
| cmrdporcupine wrote:
| I feel like people have never been _not_ talking about it. I
| probably should have tossed in a reference to Heidegger &
| his focus on 'instrumentality' etc, too.
|
| But the thing is.. how do they say it? _" Money talks,
| bullshit walks?"_
|
| People follow what fills their stomach. And
| mechanization/automation/computation, it does that in spades.
| I mean... I dropped out of my philosophy major decades ago to
| go do the More Fun To Compute thing... My bank account shows
| it.
|
| In any case... the first major uses of computers were for
| war. The next, for automation in commerce and finance. That's
| what computers are really for.
| modzu wrote:
| there are a lot of interesting and deep ideas here, about
| technology in general. but the critics of ai really need to do
| better. according to this argument, once you understand how your
| own brain works, you see human intelligence is also just an
| "illusion". fundamentally though, there is no reason a computer
| cannot think. any argument against this requires an understanding
| of the nature of thought and intelligence, in humans, which is
| quite outside the wheelhouse of most classical computer
| scientists. for a more refreshing perspective, read anything by
| hinton
| qsort wrote:
| The more simple counterargument, very well within the
| wheelhouse of any computer scientist, is that the question is
| completely irrelevant as it's entirely predicated on one's
| definition of "intelligence".
|
| Successful software solves a problem correctly and efficiently.
| It doesn't matter that GCC doesn't compile code like I would if
| I had to do that by hand, you still end up with compiled code,
| which was the original problem to solve. It doesn't matter that
| the sort() routine doesn't sort numbers like I would, it still
| does.
|
| Playing the imitation game for its own sake is kind of silly.
|
| The "illusion" lies in the attribution to a machine of
| capabilities it does not have.
| modzu wrote:
| yes, but we havent pulled off this engineering end run yet
| AlexandrB wrote:
| > fundamentally though, there is no reason a computer cannot
| think.
|
| This is true, but I think there is also a fundamental reason
| computers cannot _understand_ us. If I tell a future AGI "I am
| sick" or "I am happy", the AGI may have a _conceptual_
| understanding of what I mean, but cannot empathize with my
| condition. Many human experiences are intimately tied to our
| bodies and our hormonal systems. How do you explain a concept
| like "pain" to an intelligence that has no nerves?
| modzu wrote:
| its a great point. but then we needn't constrain this problem
| to computers, there are plenty of human experiences other
| humans cannot possibly empathize with (what do you really
| know about ptsd, autism, even different tastes in food or
| music). i think there must be ways to build something like
| understanding through analogy and other modalities, accepting
| that it is different than shared experience
| pixl97 wrote:
| But that is a different question entirely that is typically
| called the "AI alignment issue". If something understands us
| and or is aligned with our needs is completely and totally
| different if it is intelligent/conscious.
| noorkersz wrote:
| by using the cost function somehow??
| mjburgess wrote:
| > what largely transformed Weizenbaum into an outspoken critic of
| AI ... was his revelation that even once the processes were
| explained many people still bought into the "illusion." ... that
| even many people who understood the inner workings of computers
| quite well could still get swept away as well. Weizenbaum
| observed that ... "if nothing else, how easy it is to create and
| maintain the illusion of understanding, hence perhaps of
| judgement deserving of credibility... A certain danger lurks
| there."
|
| This speaks to my view _exactly_ , and I haven't seen it put
| better.
|
| The latent (imv) schizophrenic impulse to impart consciousness to
| any-old-object in the world is here being exploited by peddlers
| (often charlatans) of technology. This makes the "confirmation
| bias" here _severe_ and dangerous.
|
| People don't just _want to believe_ , their animal psychology
| primes them for (over-)imparting consciousness. This is a
| specific cognitive vulnerability that is highly pernicious.
|
| Even, trivially, consider "deresponsibilization", that process by
| which people take whatever-the-machine-says to be "The Answer".
| Why? Well, the machine "must know".
|
| Less trivially, moral bias: "presumably" the machine is
| "objective" and so is morally superior.
|
| How often do we hear this? Too much.
| rhn_mk1 wrote:
| I'm not so quick to attach a judgement to anthropomorphization.
| Well okay, I am, but not because I think it's wrong.
|
| After all, panpsychism has not been defeated as a philosophical
| standpoint [0]. Perhaps any-old-object in the world _is
| actually conscius_ , just not all of us can detect it.
|
| That the impulse to see consciousness in a thing is being used
| by charlatan is unfortunate, but has no bearing on the actual
| truth of whether the thing actually is conscious.
|
| [0]https://plato.stanford.edu/entries/panpsychism/
| mjburgess wrote:
| I think part of the scam is the move from "practical
| definitions" of words to (pseudo-) philosophical ones.
|
| There ought be a name for this type of psychological
| manipulation, but I don't know it. It is universal in a
| certain type of "motivational scam".
|
| Crypto Scammer: Isn't it bad that finance is so centralised?
|
| Crypto Scammer: Let me sell you a decentralized database!
|
| Sceptic: But that isn't the same sense of decentarlization?
|
| Scammer: Ah! But doesnt power flow from centralization of
| infrastructure?
|
| Sceptic:... maybe, maybe not. Hey, why do you have your hand
| in my pocket?
|
| Here, whether "consciousness" is on a continuum across all
| physical objects, _or_ whether only a subset of objects have
| it, is kinda irrelevant:
|
| The machine doesnt care about you, has no intentions, has
| nothing it wishes to say, and doesnt know anything about your
| or anything else. Its text generation is only a reflection of
| you, and the average of the superficial data of everyone else
| around you.
|
| In sum:
|
| Don't take philosophy classes from sales people.
| adamisom wrote:
| I'm only addressing a fraction of your many excellent points,
| but: this is actually why uncontrolled AGI is so spooky. It's
| the prospect of a future where space gets colonized-or
| _whatever_ crazy goal(s)-but there 's "nobody home", no qualia,
| no experiencer.
| adamsmith143 wrote:
| >The latent (imv) schizophrenic impulse to impart consciousness
| to any-old-object in the world is here being exploited by
| peddlers (often charlatans) of technology. This makes the
| "confirmation bias" here severe and dangerous.
|
| Have any examples of charlatans peddling "conscious" AIs?
| haswell wrote:
| Even when no direct claims about consciousness are being
| made, that is not stopping people from behaving as if they
| are conscious programs.
|
| I think we're seeing this playing out in discussions about
| generative AI with people justifying certain behaviors
| regarding the use of artist's data in training models by
| comparing the program to a human going through the process of
| "learning" and "gaining inspiration" from the work of others.
| Some people seem fully convinced that what these programs are
| doing is equivalent to human behavior to a degree that would
| qualify the program to receive human-like considerations,
| i.e. treating the software like an entity instead of just a
| program/tool to be wielded by the end-user.
|
| I think this is one of the most pernicious issues, and people
| seem to have a hard time recognizing the trap they've fallen
| into.
|
| I think this is also an outcome of this software being
| sufficiently complex to be too hard to understand without
| investing time and effort in understanding it.
|
| I think this results in the mis-attribution of intelligence
| when instead it's just a really clever piece of software. But
| the casual commenter cannot estimate the gap between "pretty
| clever" and "so complex that it's actual intelligence".
| dnissley wrote:
| > _Some people seem fully convinced that what these
| programs are doing is equivalent to human behavior to a
| degree that would qualify the program to receive human-like
| considerations, i.e. treating the software like an entity
| instead of just a program /tool to be wielded by the end-
| user._
|
| Wait, what? Who is doing this??
| pixl97 wrote:
| But I think in many ways you're falling into the same
| trap...
|
| So, let me set the first trap for you. Give me a
| logically/mathematically defined definition of intelligence
| and one of consciousness. Unfortunately after decades of
| back and forth on this we typically end up with answers
| that fall out of science and into the "I'll know it when I
| see it territory".
|
| If you are going to describe machine
| intelligence/consciousness your definition must be able to
| cover intelligent behaviors we see from single cell life to
| the massive complexity we see in humans. Attempting to
| handwave and say "it's not as complex as humans therefore
| it's not intelligent/conscious" is a complete and total
| failure from my point of view.
| burnished wrote:
| I think you are misreading that argument. If I understand
| correctly the point is that you are already allowed to look
| at and make derivative works of art - the machine version
| of that is not fundamentally any different, especially
| since it is not reproducing works in whole but rather
| reproducing a 'style'.
|
| That isnt an endorsement of the argument, my point is that
| you dont have to believe the black box has any independent
| intelligence to draw an analogy between what it does and
| what we already allow.
| eointierney wrote:
| Anthropomorphisation is a curse, eh?
|
| We've seen this in deities for as long as we've been around,
| royalties then took the crown, later subsumed by governments,
| then the daft notion of non-human personalities called
| companies. Now, aproximate intelligence systems.
|
| We humans are credulous
___________________________________________________________________
(page generated 2023-01-27 23:01 UTC)