[HN Gopher] Systems design 2: What we hope we know
___________________________________________________________________
Systems design 2: What we hope we know
Author : Amorymeltzer
Score : 103 points
Date : 2023-04-15 14:10 UTC (8 hours ago)
(HTM) web link (apenwarr.ca)
(TXT) w3m dump (apenwarr.ca)
| carapace wrote:
| FWIW, apenwarr is one of the most intelligent humans I've ever
| met. When people bring up 10x programmers I think of him. I don't
| really have much to add, I just wanted to mention that for
| context.
| tudorw wrote:
| thanks, a good read, I'm always curious to go back and look at
| early knowledge system innovators who were looking at the
| 'causes' of 'breakthrough hypothesis' and looking to then
| 'engineer' a surrounding system to support more frequent
| breakthroughs.
| rektide wrote:
| Placing emerged complexity/magic in a bucket together & saying
| science despises them is a novel & revelatory point of view.
| Avery is such a treasure!
|
| Lovely idea towards the bottom too,
|
| > _Or maybe we need to draw a distinction between_ producing
| _insight and_ recognizing _insight._
|
| I personally identify with this & see it as super core a
| distinction. I'm reasonably bright & can find some clever ways of
| doing things even if I haven't any learnings here. But like 90%
| of what I love & appreciate about self is a resourcefulness &
| ability to recognize & integrate insight.
| kqr wrote:
| On the quote opening the article:
|
| > Any sufficiently advanced technology is indistinguishable from
| magic.
|
| I had a lecturer who used to counter "if something ever seems
| like magic it means you're not asking enough questions."
|
| That stuck with me, although the article goes on to counter it
| fairly well.
| nyanpasu64 wrote:
| > In short, in software engineering, we acknowledge that failures
| happen and we measure them, characterize them, and compensate for
| them. We don't aim for perfection.
|
| If software engineering revolves around writing software complex
| enough to have bugs not understood, or accepting memory safety
| errors as a natural byproduct of programming (rather than
| programming mistakes to be exterminated on sight, and made
| impossible by design unless it prevents writing useful programs),
| I'd be more effective and happier practicing applied math than
| engineering.
| skybrian wrote:
| It's also accepting that performance is variable, that the
| network is not reliable, that machines can fail at any time,
| and that memory corruption happens.
|
| When you build systems at scale you need to worry about these
| things.
|
| Programming language API boundaries don't make real-time
| performance guarantees, not even in Rust. We hope to get
| typical performance. In a mathematical context (for example,
| proving programs correct), you often ignore these things and
| worry only about whether the program will give you the right
| answer, eventually, when the hardware works.
| danielvaughn wrote:
| "I've never seen anyone say ChatGPT feels like magic"
|
| This is a curious line - I have, and I personally have that
| reaction when I use it. It might even be the most magical
| technology I've ever encountered.
| jt2190 wrote:
| > This is a curious line...
|
| Literally in the _next sentence_ the author also remarks that
| he finds it odd:
|
| > "That makes me curious."
|
| He finds it curious because he also finds the technology is
| magical.
|
| Eventually he concludes:
|
| > "Let's say you have a Large Language Model AI, trained on all
| the text in the world. We understand the mechanisms
| (engineering) but the outcomes are surprising and emergent
| (magic). Its job is to repeatedly predict the next word that
| would come if a person wrote it - a form of negative latency,
| or non-causality. From this, for reasons we don't really
| understand, it can say an impressive variety of unexpectedly
| meaningful things."
|
| > "When you have all the text in the world memorized, that
| means you have access to all the insights that have ever been
| written. You need only recognize them, and have a good idea of
| what the reader knows already, and you can produce insights -
| things the reader has never heard before - on demand. You can
| get an A+ from my English teacher. Can our AI do that? I don't
| know. Certainly sometimes, maybe not reliably yet. Any time it
| does, it will feel like magic."
| amadeuspagel wrote:
| I guess it doesn't feel like magic because it feels human?
| MH15 wrote:
| While I personally don't think ChatGPT feels magical, I know
| many people who feel this way. It's a really common opinion,
| especially among non-programmers.
| Sniffnoy wrote:
| > Since we now know what magic means, we know what magical
| thinking means: a tendency to interpret something as magic
| instead of trying to understand it.
|
| No, "magical thinking" has a particular meaning, and it's not
| that: https://en.wikipedia.org/wiki/Magical_thinking
|
| Mind you, people often use the phrase in a pretty loose way as a
| way of dismissing an argument, but it does seem to be based on
| the technical use, and not the meaning Apenwarr puts forth.
| epgui wrote:
| You're allowed to define different meanings for words if it
| makes your argument go forward and if you make it clear what it
| is.
|
| However I still find the argument very unpersuasive, and the
| usual meaning of "magical thinking" makes me feel uncomfortable
| too.
| Sniffnoy wrote:
| Sure, but in this case he's explicitly discussing other
| people's use:
|
| > On the other hand, people who believe AIs and specifically
| LLMs possess "intelligence" are often accused of "magical
| thinking."
| singpolyma3 wrote:
| That definition seems pretty closely related to his.
| epgui wrote:
| I appreciate that the author is using very specific meaning for
| the term "magical thinking", but even with this, I am
| uncomfortable with the assertion that somehow understanding
| emergence has anything to do with magic or magical thinking, as
| (re)defined in the article.
|
| Anyone who spends enough time studying biochem will have an
| intuitive understanding / acceptance that biology emerges from
| chemistry, which emerges from physics... But there's not really
| any "magical thinking" (as defined in the article) involved.
|
| Emergence of cognitive processes and experiences in the brain is
| not really any different in this respect. If anything, it's
| people who insist that human experiences are "special" and "just
| different, obviously" that are doing the magical thinking.
___________________________________________________________________
(page generated 2023-04-15 23:00 UTC)