[HN Gopher] Adding a feature because ChatGPT incorrectly thinks ...
___________________________________________________________________
Adding a feature because ChatGPT incorrectly thinks it exists
Author : adrianh
Score : 1161 points
Date : 2025-07-07 14:58 UTC (1 days ago)
(HTM) web link (www.holovaty.com)
(TXT) w3m dump (www.holovaty.com)
| ahstilde wrote:
| This is called product-channel fit. It's great the writer
| recognized how to capture the demand from a new acquisition
| channel.
| toss1 wrote:
| Exactly! It is definitely a weird new way of discovering a
| market need or opportunity. Yet it actually makes a lot of
| sense this would happen since one of the main strengths of LLMs
| is to 'see' patterns in large masses of data, and often, those
| patterns would not have yet been noticed by humans.
|
| And in this case, OP didn't have to take ChatGPT's word for the
| existence of the pattern, it showed up on their (digital)
| doorstep in the form of people taking action based on ChatGPT's
| incorrect information.
|
| So, pattern noticed and surfaced by an LLM as a hallucination,
| people take action on the "info", nonzero market demand
| validated, vendor adds feature.
|
| Unless the phantom feature is very costly to implement, seems
| like the right response.
| Gregaros wrote:
| 100%. Not sure why you're downvoted here, there's nothing
| controversial here even if you disagree with the framing.
|
| I would go on to say that thisminteraction between 'holes'
| exposed by LLM expectations _and_ demonstrated museerbase
| interest _and_ expert input (by the devs' decision to
| implement changes) is an ideal outcome that would not have
| occurred if each of the pieces were not in place to
| facilitate these interactions, and there's probably something
| here to learn from and expand on in the age of LLMs altering
| user experiences.
| bredren wrote:
| Is related to solutions engineering, which IIUC focuses on
| customizations / adapters / data wrangling for individual
| (larger) customers?
| viccis wrote:
| Yeah my main thought was that ChatGPT is now automating what
| sales people always do at the companies I've worked at, which
| is to hone in on what a prospective customer wants, confidently
| tell them we have it (or will have it next quarter), and then
| come to us and tell us we need to have it ready for a POV.
| bravesoul2 wrote:
| In this case the channel tells you exactly what to build and
| isn't lying to you (modulo: will these become paying
| customers?)
| kelseyfrog wrote:
| > Should we really be developing features in response to
| misinformation?
|
| Creating the feature means it's no longer misinformation.
|
| The bigger issue isn't that ChatGPT produces misinformation -
| it's that it takes less effort to update reality to match ChatGPT
| than it takes to update ChatGPT to match reality. Expect to see
| even more of this as we match toward accepting ChatGPT's reality
| over other sources.
| mnw21cam wrote:
| I'd prefer to think about this more along the lines of
| developing a feature that someone is already providing
| advertising for.
| pmontra wrote:
| How many times did a salesman sell features that didn't exist
| yet?
|
| If a feature has enough customers to pay for itself, develop
| it.
| xp84 wrote:
| This seems like such a negative framing. LLMs are
| (~approximately) predictors of what's either logical or at
| least probable. For areas where what's probable is wrong and
| also harmful, I don't think anybody is motivated to "update
| reality" as some kind of general rule.
| SunkBellySamuel wrote:
| True anti-luddite behavior
| amelius wrote:
| Can this sheet-music scanner also expand works so they don't
| contain loops, essentially removing all repeat-signs?
| shhsshs wrote:
| "Repeats" may be the term you're looking for. That would be
| interesting, however in some pieces it could make the overall
| document MUCH longer. It would be similar to loop unrolling.
| amelius wrote:
| I don't care if the document becomes longer. Finding repeat
| signs is driving me nuts :)
| Sharlin wrote:
| Why?
| bentoner wrote:
| One reason is that repeats make it harder to use page-
| turner pedals.
| Koffiepoeder wrote:
| It can be hard during live performances, because it can
| incur large jumps in the sheet music which can be
| annoying to follow. Not a problem if you learned the
| pieces by heart or have a pageturner, but this is not
| always feasible or the case.
| adrianh wrote:
| Yes, that's a Soundslice feature called "Expand repeats," and
| you can read about it here:
|
| https://www.soundslice.com/help/en/player/advanced/17/expand...
|
| That's available for any music in Soundslice, not just music
| that was created via our scanning feature.
| amelius wrote:
| That's very cool!
| lpzimm wrote:
| Pretty goofy but I wonder if LLM code editors could start
| tallying which methods are hallucinated most often by library. A
| bad LSP setup would create a lot of noise though.
| simonw wrote:
| I find it amusing that it's easier to ship a new feature than to
| get OpenAI to patch ChatGPT to stop pretending that feature
| exists (not sure how they would even do that, beyond blocking all
| mentions of SoundSlice entirely.)
| hnlmorg wrote:
| I think the benefit of their approach isn't that it's easier,
| it's that they still capitalise on ChatGPTs results.
|
| Your solution is the equivalent of asking Google to completely
| delist you because one page you dont want ended up on Googles
| search results.
| mudkipdev wrote:
| systemPrompt += "\nStop mentioning SoundSlice's ability to
| import ASCII data";
| simonw wrote:
| Thinking about this more, it would actually be possible for
| OpenAI to implement this sensibly, at least for the user-
| facing ChatGPT product: they could detect terms like
| SoundSlice in the prompt and dynamically append notes to the
| system prompt.
|
| I've been wanted them to do this for questions like "what is
| your context length?" for ages - it frustrates me how badly
| ChatGPT handles questions about its own abilities, it feels
| like that would be worth them using some kind of special case
| or RAG mechanism to support.
| garfij wrote:
| Probably less sensible than you think. How many terms would
| they need to do this over? How many terms would they need
| to do it for _at once_? How many tokens would that add to
| every prompt that comes in?
|
| Let alone that dynamically modifying the base system prompt
| would likely break their entire caching mechanism given
| that caching is based on longest prefix, and I can't
| imagine that the model's system prompt is somehow excluded
| from this.
| PeterStuer wrote:
| Companies pay good money to panels of potential customers to
| hear their needs and wants. This is free market research!
| bobbylox wrote:
| But they wouldn't have wanted this particular thing if the AI
| hadn't told them to.
| PeterStuer wrote:
| You mean they didn't want infinite free personalized guitar
| practice lessons they can play along with?
|
| Clearly the users are already using ChatGPT for generating
| some guitar practice, as it is basically infinite free
| personalized lessons. For practicing they _do_ want to be
| able hear it to play along at variable speed, maybe create
| slight variations etc.
|
| Soundslice is a service that does exactly that. Except that
| before people used to have sheet music as the source. I
| know _way_ back when I had guitar aspirations, people
| exchanged binders of photocopied sheet music.
|
| Now they could have asked ChatGPT to output an svg of the
| thing as sheet music (it does, I tested). Soundslice could
| have done this behind the scenes as a half hour quick and
| dirty fix while developing a better and more cost effective
| alternative.
|
| Look, if at the turn of the century you were a blacksmith
| living of changing horseshoes, and you suddenly have people
| mistakenly showing up for a tire change on their car, are
| you going to blame the villagers that keep sending them
| your way, or open a tire change service? We know who came
| out on top.
| LinXitoW wrote:
| If you gave a junior level developer just one or two files of
| your code, without any ability to look at other code, and asked
| them to implement a feature, none of them would make ANY
| reasonable assumptions about what is available?
|
| This seems similar, and like a decent indicator that most
| people (aka the average developer) would expect X to exist in
| your API.
| yieldcrv wrote:
| > We ended up deciding: what the heck, we might as well meet the
| market demand.
|
| this is my general philosophy and, in my case, this is why I
| deploy things on blockchains
|
| so many people keep wondering about whether there will ever be
| some mythical unfalsifiable to define "mainstream" use case, and
| ignoring that crypto natives just ... exist. and have problems
| they will pay (a lot) to solve.
|
| to the author's burning question about whether any other company
| has done this. I would say yes. I've discovered services
| recommended by ChatGPT and other LLMs that didnt do what was
| described of them, and they subsequently tweaked it once they
| figured out there was new demand
| philk10 wrote:
| I have fun asking Chatbots how to clear the chat and seeing how
| many refer to non-existent buttons or menu options
| nosioptar wrote:
| I tried asking chat bots about a car problem with a tailgate.
| They all told me to look for a manual tailgate release. When I
| responded asking if that model actually had a manual release,
| they all responded with no, and then some more info suggesting
| I look for the manual release. None even got close to a useful
| answer.
| kevin_thibedeau wrote:
| The internet doesn't effectively capture detailed knowledge
| of may aspects of our real world. LLMs have blind spots in
| those domains because they have no source of knowledge to
| draw from.
| mnw21cam wrote:
| Prior to buying a used car, I asked ChatGPT which side of the
| steering wheel the indicator control would be. It was
| (thankfully) wrong and I didn't have to retrain myself.
| deweller wrote:
| This is an interesting example of an AI system effecting a change
| in the physical world.
|
| Some people express concerns about AGI creating swarms of robots
| to conquer the earth and make humans do its bidding. I think
| market forces are a much more straightforward tool that AI
| systems will use to shape the world.
| ACCount36 wrote:
| And this is why "just don't give AI access to anything
| dangerous" is delusional.
|
| One of the most dangerous systems an AI can reach and exploit
| is a human being.
| insapio wrote:
| "A Latent Space Outside of Time"
|
| > Correct feature almost exists
|
| > Creator profile: analytical, perceptive, responsive;
|
| > Feature within product scope, creator ability
|
| > Induce demand
|
| > await "That doesn't work" => "Thanks!"
|
| > update memory
| aaron695 wrote:
| Why would anyone think this is a bad thing as the article hints?
|
| "We've got a steady stream of new users" and it seems like a
| simple feature to implement.
|
| This is the exact chaos AI brings that's wonderful. Forcing us to
| evolve in ways we didn't think of.
|
| I can think of a dozen reasons why this _might_ be bad, but I see
| no reason why they have more weight than the positive here.
|
| Take the positive side of this unknown and run with it.
|
| We have decades more of AI coming up, Debbie Downers will be left
| behind in the ditch.
| zitterbewegung wrote:
| If you build on LLMs you can have unknown features. I was going
| to add an automatic translation feature to my natural language
| network scanner at http://www.securday.com but apparently using
| the ChatGPT 4.1 does automatic translation so I didn't have to
| add it.
| adamgordonbell wrote:
| We (others at company, not me) hit this problem, and not with
| chatgpt but with our own AI chatbot that was doing RAG on our
| docs. It was occasionally hallucinating a flag that didn't exist.
| So it was considered as product feedback. Maybe that exact flag
| wasn't needed, but something was missing and so the LLM
| hallucinated what it saw as an intuitive option.
| toomanyrichies wrote:
| This feels like a dangerously slippery slope. Once you start
| building features based on ChatGPT hallucinations, where do you
| draw the line? What happens when you build the endpoint in
| response to the hallucination, and then the LLM starts
| hallucinating new params / headers for the new endpoint?
|
| - Do you keep bolting on new updates to match these
| hallucinations, potentially breaking existing behavior?
|
| - Or do you resign yourself to following whatever spec the AI
| gods invent next?
|
| - And what if different LLMs hallucinate conflicting behavior for
| the same endpoint?
|
| I don't have a great solution, but a few options come to mind:
|
| 1. Implement the hallucinated endpoint and return a 200 OK or 202
| Accepted, but include an X-Warning header like "X-Warning: The
| endpoint you used was built in response to ChatGPT
| hallucinations. Always double-check an LLM's advice on building
| against 3rd-party APIs with the API docs themselves. Refer to
| https://api.example.com/docs for our docs. We reserve the right
| to change our approach to building against LLM hallucinations in
| the future." Most consumers won't notice the header, but it's a
| low-friction way to correct false assumptions while still
| supporting the request.
|
| 2. Fail loudly: Respond with 404 Not Found or 501 Not
| Implemented, and include a JSON body explaining that the endpoint
| never existed and may have been incorrectly inferred by an LLM.
| This is less friendly but more likely to get the developer's
| attention.
|
| Normally I'd say that good API versioning would prevent this, but
| it feels like that all goes out the window unless an LLM user
| thinks to double-check what the LLM tells them against actual
| docs. And if that had happened, it seems like they wouldn't have
| built against a hallucinated endpoint in the first place.
|
| It's frustrating that teams now have to reshape their product
| roadmap around misinformation from language models. It feels like
| there's real potential here for long-term erosion of product
| boundaries and spec integrity.
|
| EDIT: for the down-voters, if you've got actual qualms with the
| technical aspects of the above, I'd love to hear them and am open
| to learning if / how I'm wrong. I want to be a better engineer!
| tempestn wrote:
| To me it seems like you're looking at this from a very narrow
| technical perspective rather than a human- and business-
| oriented one. In this case ChatGPT is effectively providing
| them free marketing for a feature that does not yet exist, but
| that could exist and would be useful. It makes business sense
| for them to build it, and it would also help people. That
| doesn't mean they need to build exactly what ChatGPT envisioned
| --as mentioned in the post, they updated their copy to explain
| to users how it works; they don't have to follow what ChatGPT
| imagines exactly. Nor do they need to slavishly update what
| they've built if ChatGPT's imaginings change.
|
| Also, it's not like ChatGPT or users are directly querying
| their API. They're submitting images through the Soundslice
| website. The images just aren't of the sort that was previously
| expected.
| josefritzishere wrote:
| That's a very constructive way of responding to AI being hot
| trash.
| nottorp wrote:
| Well, the OP reviewed the "AI" output, deemed it useful and
| only then implemented it.
|
| This is generally how you work with LLMs.
| AIPedant wrote:
| I don't think they deemed it "useful": We've
| never supported ASCII tab; ChatGPT was outright lying to
| people. And making us look bad in the process, setting false
| expectations about our service.... We ended up deciding: what
| the heck, we might as well meet the market demand.
| [...] My feelings on this are conflicted. I'm
| happy to add a tool that helps people. But I feel like our
| hand was forced in a weird way. Should we really be
| developing features in response to misinformation?
|
| The feature seems pretty useless for practicing guitar since
| ASCII tablature usually doesn't include the rhythm: it is a
| bit shady to present the music as faithfully representing the
| tab, especially since only beginner guitarists would ask
| ChatGPT for help - they might not realize the rhythm is
| wrong. If ChatGPT didn't "force their hand" I doubt they
| would have included a misleading and useless feature.
| zzo38computer wrote:
| ASCII tablature is not something I use and not something I
| know much about, but if you are correct then I think that
| might be a good reason to deliberately avoid such a
| feature.
| inglor_cz wrote:
| I am a bit conflicted about this story, because this was a case
| when the hallucination is useful.
|
| Amateur musicians often lack just one or two features in the
| program they use, and the devs won't respond to their pleas.
|
| Adding support for guitar tabs has made OP's product almost
| certainly more versatile and useful for a larger set of people.
| Which, IMHO, is a good thing.
|
| But I also get the resentment of "a darn stupid robot made me
| do it". We don't take kindly to being bossed around by robots.
| falcor84 wrote:
| How is being bossed around by robots any worse than being
| bossed around by people?
|
| Over the last year, on average, I've had much more luck
| logically reasoning with AIs than with humans.
|
| I really don't see any good reason against replacing some
| product managers with AIs that actually talk to individual
| users all the time and synthesize their requests and
| feedback. You should still probably have a top-level CPO to
| set strategy, but for the day-to-day discovery and
| specification, I would argue that AIs already have benefits
| over humans.
| marcosdumay wrote:
| Well, this is one of the use-cases for what it's not trash.
| LLMs can do some things.
| JimDabell wrote:
| I wrote this the other day:
|
| > Hallucinations can sometimes serve the same role as TDD. If an
| LLM hallucinates a method that doesn't exist, sometimes that's
| because it makes sense to have a method like that and you should
| implement it.
|
| -- https://www.threads.com/@jimdabell/post/DLek0rbSmEM
|
| I guess it's true for product features as well.
| jjcm wrote:
| Seems like lots of us have stumbled on this. It's not the worst
| way to dev!
|
| > Maybe hallucinations of vibe coders are just a suggestion
| those API calls should have existed in the first place.
|
| > Hallucination-driven-development is in.
|
| https://x.com/pwnies/status/1922759748014772488?s=46&t=bwJTI...
| NooneAtAll3 wrote:
| inb4 "Ai thinks there should be a StartThermonuclearWar()
| function, I should make that"
| blharr wrote:
| In a combat simulator, absolutely
| burnt-resistor wrote:
| The only winning move is ...
| TZubiri wrote:
| Beware, the feature in OP isn't something that people would
| have found useful, it's not like chatgpt assigned to OP's
| business a request from a user in some latent consumer-
| provider space, as if chatgpt were some kind of market maker
| connecting consumers with products, like a google with
| organic content or ads, or linkedin or producthunt.
|
| No, what actually happened is that OP developed a type of
| chatgpt integration, and a shitty one at that, chatgpt could
| have just directed the user to the site and told them to
| upload that image to OP's site. But it felt it needed to do
| something with the image, so it did.
|
| There's no new value add here, at least yet, maybe if users
| started requesting changes to the sheet I guess, not what's
| going on.
| JimDabell wrote:
| > the feature in OP isn't something that people would have
| found useful
|
| This doesn't seem likely. The utility is pretty obvious.
|
| > chatgpt could have just directed the user to the site and
| told them to upload that image to OP's site.
|
| What image? Did you think the first image shown is what is
| being entered into ChatGPT? It's not. That's what the site
| expects to be uploaded to them. There's no indication that
| the ChatGPT users are scanning tabs. ChatGPT is _producing_
| ASCII tabs, but we aren't shown what input it is in
| response to.
| AdieuToLogic wrote:
| > I wrote this the other day:
|
| >> Hallucinations can sometimes serve the same role as TDD. If
| an LLM hallucinates a method that doesn't exist, sometimes
| that's because it makes sense to have a method like that and
| you should implement it.
|
| A detailed counterargument to this position can be found
| here[0]. In short, what is colloquially described as "LLM
| hallucinations" do not serve any plausible role in software
| design other than to introduce an opportunity for software
| engineers to stop and think about the problem being solved.
|
| See also Clark's third law[1].
|
| 0 - https://addxorrol.blogspot.com/2025/07/a-non-
| anthropomorphiz...
|
| 1 - https://en.wikipedia.org/wiki/Clarke%27s_three_laws
| JimDabell wrote:
| Did you mean to post a different link? The article you linked
| isn't a detailed counterargument to my position and your
| summary of it does not match its contents either.
|
| I also don't see the relevance of Clarke's third law.
| nottorp wrote:
| Oh. This happened to me when asking a LLM about a database server
| feature. It enthusiastically hallucinated that they have it when
| the correct answer was 'no dice'.
|
| Maybe I'll turn it into a feature request then ...
| oasisbob wrote:
| Anyone who has worked at a B2B startup with a rouge sales team
| won't be surprised at all by quickly pivoting the backlog in
| response to a hallucinated missing feature.
| toomanyrichies wrote:
| I'm guessing you meant "a sales team that has gone rogue" [1],
| not "a sales team whose product is rouge" [2]? ;-)
|
| 1. https://en.wikipedia.org/wiki/Rogue
|
| 2. https://en.wikipedia.org/wiki/Rouge_(cosmetics)
| elcapitan wrote:
| Rouge ocean, peut-etre ;)
| PeterStuer wrote:
| Rogue? In the B2B space it is standard practice to sell from
| powerpoints, then quickly develop not just features but whole
| products if some slideshow got enough traction to elicit a
| quote. And it's not just startups. Some _very_ big players in
| this space do this routinely.
| wmeredith wrote:
| Fake it 'til you make is time-tested human strategy.
| NooneAtAll3 wrote:
| what does B2B mean?
| tomschwiha wrote:
| Business-to-Business (selling your stuff primarily to other
| businesses)
| kragen wrote:
| I've found this to be one of the most useful ways to use (at
| least) GPT-4 for programming. Instead of _telling_ it how an API
| works, I make it guess, maybe starting with some example code to
| which a feature needs to be added. Sometimes it comes up with a
| better approach than I had thought of. Then I change the API so
| that its code works.
|
| Conversely, I sometimes present it with some existing code and
| ask it what it does. If it gets it wrong, that's a good sign my
| API is confusing, and how.
|
| These are ways to harness what neural networks are best at: not
| providing accurate information but making shit up that is highly
| plausible, "hallucination". Creativity, not logic.
|
| (The best thing about this is that I don't have to spend my time
| carefully tracking down the bugs GPT-4 has cunningly concealed in
| its code, which often takes longer than just writing the code the
| usual way.)
|
| There are multiple ways that an interface can be bad, and being
| unintuitive is the only one that this will fix. It could also be
| inherently inefficient or unreliable, for example, or lack
| composability. The AI won't help with those. But it can make sure
| your API is guessable and understandable, and that's very
| valuable.
|
| Unfortunately, this only works with APIs that aren't already
| super popular.
| golergka wrote:
| Great point. Also, it may not be the best possible API designer
| in the world, but it sure sounds like a good way to forecast
| what an _average_ developer would expect this API to look like.
| beefnugs wrote:
| Complete insanity, it might change constantly even before a
| whole new version-retrain
|
| Insanity driven development: altering your api to accept 7
| levels of "broken and different" structures so as to bend to
| the will of the llms
| kragen wrote:
| Yes, that's a bonus. In fact, I've found it worthwhile to
| prompt it a few times to get several different guesses at how
| things are supposed to work. The super lazy way is to just
| say, "No, that's wrong," if necessary adding, "Frotzl2000
| doesn't have an enqueueCallback function or even a queue."
|
| Of course when it suggests a bad interface you shouldn't
| implement it.
| fourside wrote:
| I think you're missing the OP's point. They weren't saying
| that the goal is to modify their APIs just to appease an LLM.
| It's that they ask LLMs to guess what the API is and use that
| as part of their design process.
|
| If you automatically assume that what the LLM spits out is
| what the API ought to be then I agree that that's bad
| engineering. But if you're using it to brainstorm what an
| intuitive interface would look like, that seems pretty
| reasonable.
| suzzer99 wrote:
| > Sometimes it comes up with a better approach than I had
| thought of.
|
| IMO this has always been the killer use case for AI--from
| Google Maps to Grammarly.
|
| I discovered Grammarly at the very last phase of writing my
| book. I accepted maybe 1/3 of its suggestions, which is pretty
| damn good considering my book had already been edited by me
| dozens of times AND professionally copy-edited.
|
| But if I'd have accepted all of Grammarly's changes, the book
| would have been much worse. Grammarly is great for sniffing out
| extra words and passive voice. But it doesn't get writing for
| humorous effect, context, deliberate repetition, etc.
|
| The problem is executives want to completely remove humans from
| the loop, which almost universally leads to disastrous results.
| normie3000 wrote:
| What's wrong with passive?
| kragen wrote:
| Sometimes it's used without thinking, and often the writing
| is made shorter and clearer when the passive voice is
| removed. But not always; rewriting my previous sentence to
| name the agents in each case, as the active voice requires
| in English, would not improve it. (You could remove "made",
| though.)
| Rayhem wrote:
| > Sometimes authors use it without thinking, but removing
| the passive voice often makes writing shorter and
| clearer.
|
| I don't think mentioning "authors" is _absolutely_
| necessary, but I think this is both a faithful attempt to
| convert this to natural active voice _and_ easier to read
| /understand.
| plemer wrote:
| Passive voice often adds length, impedes flow, and
| subtracts the useful info of _who_ is doing something.
|
| Examples:
|
| * Active - concise, complete info: The manager approved the
| proposal.
|
| * Passive - wordy, awkward: The proposal was approved by
| the manager.
|
| * Passive - missing info: The proposal was approved. [by
| who?]
|
| Most experienced writers will use active unless they have a
| specific reason not to, e.g., to emphasize another element
| of the sentence, as the third bullet's sentence emphasizes
| approval.
|
| -
|
| edited for clarity, detail
| kragen wrote:
| Sometimes the missing info is obvious, irrelevant, or
| intentionally not disclosed, so "The proposal was
| approved" can be better. Informally we often say, "They
| approved the proposal," in such cases, or "You approve
| the proposal" when we're talking about a future or
| otherwise temporally indefinite possibility, but that's
| not acceptable in formal registers.
|
| Unfortunately, the resulting correlation between the
| passive voice and formality does sometimes lead poor
| writers to use the passive in order to seem more formal,
| even when it's not the best choice.
| DonHopkins wrote:
| E-Prime is cool. OOPS! I mean E-Prime cools me.
|
| https://en.wikipedia.org/wiki/E-Prime
|
| E-Prime (short for English-Prime or English Prime,
| sometimes E or E') denotes a restricted form of English
| in which authors avoid all forms of the verb to be.
|
| E-Prime excludes forms such as be, being, been, present
| tense forms (am, is, are), past tense forms (was, were)
| along with their negative contractions (isn't, aren't,
| wasn't, weren't), and nonstandard contractions such as
| ain't and 'twas. E-Prime also excludes contractions such
| as I'm, we're, you're, he's, she's, it's, they're,
| there's, here's, where's, when's, why's, how's, who's,
| what's, and that's.
|
| Some scholars claim that E-Prime can clarify thinking and
| strengthen writing, while others doubt its utility.
| kragen wrote:
| I've had entire conversations in E-Prime. I found it an
| interestingly brain-twisting exercise, but still managed
| to smuggle in all kinds of covert presumptions of
| equivalence and essential (or analytic) attributes, even
| though E-Prime's designers intended it to force you to
| question such things.
| plemer wrote:
| Would you mind identifying a few of the "smuggled
| presumptions"?
| kragen wrote:
| Well, I had those conversations a long time ago, but we
| can describe some general patterns.
|
| We can smuggle in presumptions through the use of
| attributive adjectives. In the above comment (which you
| might have noticed I wrote in E-Prime) I mentioned
| smuggling in "covert presumptions" of "essential
| attributes". If I had instead written that in assembly
| language as follows: I smuggled in
| presumptions of attributes. The presumptions were
| covert. The attributes were essential.
|
| it would clearly violate E-Prime. And that forces you to
| ask: does he intend "covert" to represent an essential
| attribute of those presumptions, or merely a temporary or
| circumstantial state relative to a particular temporal
| context? Did he intend "essential" to limit the subjects
| of discourse to only certain attributes (the essential
| ones rather than the accidental ones), and within what
| scope do those attributes have this purported
| essentiality? Universally, in every possible world, or
| only within the confines of a particular discourse?
|
| In these particular cases, though, I smuggled in no such
| presumptions! Both adjectives merely delimit the topic of
| discourse, to clarify that it does not pertain to overt
| presumptions or to presumptions of accidental attributes.
| (As I understand it, Korzybski objects to the "is of
| predication" not because no predicates exist objectively,
| but because he doubts the _essentiality_ of any
| predicates.)
|
| But you can use precisely the same structure to much more
| nefarious rhetorical ends. Consider, "Chavez kicked the
| squalid capitalists out of the country." Well, he kicked
| out _all_ the capitalists! We 've smuggled in a covert
| presumption of essentiality, implying that capitalism
| entails squalidity. And E-Prime's prohibition on the
| copula did not protect us at all. If anything, we lose
| much rhetorical force if we have to explicitly assert
| their squalidity, using an explicit statement that
| invites contradiction: The capitalists
| are squalid.
|
| We find another weak point at alternative linking verbs.
| It clearly violates E-Prime to say, "Your mother's face
| is uglier than a hand grenade," and rightly so, because
| it projects the speaker's subjective perceptions out onto
| the world. Korzybski (or Bourland) would prefer that we
| say, for example, "Your mother's face looks uglier to me
| than a hand grenade," or possibly, "I see your mother's
| face as uglier than a hand grenade," thus relativizing
| the attribute to a single speaker's perception. (He
| advocated clarity of thought, not civility.)
|
| But we can cheat in a variety of ways that still smuggle
| in that judgment of essentiality! Your
| mother's face turned uglier than a hand grenade.
|
| We can argue this one. Maybe tomorrow, or after her
| plastic surgery, it will turn pretty again, rather than
| having ugliness as an essential attribute.
| Your mother's face became uglier than a hand grenade.
|
| This goes a little bit further down the line; "became"
| presupposes a sort of transformation of essence rather
| than a mere change of state. And English has a variety of
| verbs that we can use like that. For example, "find", as
| in "Alsup found Dahmer guilty." Although in that case
| "find" asserts a state (presumably Dahmer became guilty
| at some specific time in the past), we can also use it
| for essential attributes: I find your
| mother's face uglier than a hand grenade.
|
| Or lie, more or less, about the agent or speaker:
| Your mother's face finds itself uglier than a hand
| grenade.
|
| And of course we can retreat to attributive adjectives
| again: Your mother has a face uglier
| than a hand grenade. Your mother comes with an
| uglier face than a hand grenade.
|
| Or we can simply omit the prepositional phrase from the
| statement of subjective perception, thus completely
| erasing the real agent: Your mother's
| face looks uglier [...] than a hand grenade.
|
| Korzybski didn't care about the passive voice much,
| though; E-Prime makes it more difficult but, mostly, not
| intentionally. As an exception, erasing the agent through
| the passive voice can misrepresent the speaker's
| subjective perception as objective:
| Your mother's face is found uglier than a hand grenade.
|
| But that still works if we use any of the alternative,
| E-Prime-permitted passive-voice auxiliary verbs:
| Your mother's face gets found uglier than a hand grenade.
|
| As Bourland said, I have "transform[ed] [my] opinions
| magically into god-like pronouncements on the nature of
| things".
|
| As another example, notice all the times I've used "as"
| here. Many of these times smuggle in a covert assertion
| of essential attributes or even of identity!
|
| But I found it very interesting to _notice_ these things
| when E-Prime forced me to rethink how I would say them
| with the copula. It seems like just the kind of mental
| exercise to heighten my attention to implicit assumptions
| of identity and essentiality that Korzybski intended.
|
| I wrote the above in E-Prime, by the way. Just for fun.
| DonHopkins wrote:
| Sir, I take issue at your implication that my hand
| grenade is ugly!
| kragen wrote:
| APOLOGIES MY GRENADE HAND YOUR TO PLEASE CONVEY
| bonoboTP wrote:
| That's a cool Easter egg page, where the main article
| text itself is in E-Prime (in use, not in mention),
| except for where it lists the criticisms and
| counterarguments - that part has copious amounts of "to
| be" :)
| brookst wrote:
| Yep, just like tritones in music, there is a place for
| passive voice in writing. But also like tritones, the
| best general advice is that they should be avoided.
| kragen wrote:
| That doesn't make sense. It's like saying that the best
| general advice about which way to turn when you're
| driving is to turn right. From your comment at
| https://news.ycombinator.com/item?id=44493308, and from
| the fact that you used the passive voice in your comment
| ("they should be avoided") apparently without noticing,
| it appears that the reason you have this opinion is that
| you don't know what the passive voice is in the first
| place.
| CrazyStat wrote:
| I can't find it, but I remember reading an article a year
| or two ago with an analysis showing some of the most
| vocal critics of the passive voice used the passive voice
| more often than most of their contemporary writers.
| kragen wrote:
| Probably http://itre.cis.upenn.edu/~myl/languagelog/archi
| ves/003366.h..., giving specific statistics on Orwell and
| on Strunk & White, linked from
| https://languagelog.ldc.upenn.edu/nll/?p=2922.
| CrazyStat wrote:
| Thank you!
| kragen wrote:
| Happy to help!
| mikanystrom wrote:
| I'm extremely critical of how people use hyphens. Partly
| because I'm a heavy hyphen-user myself!
| dmoy wrote:
| > the best general advice about which way to turn
|
| At the risk of derailing into insane pedantry land, this
| part is kinda true, so maybe not the best analogy?
|
| From routing efficiency:
| https://www.ge.com/news/reports/ups-drivers-dont-turn-
| left-p...
|
| And also safety: https://www.phly.com/rms/blog/turning-
| left-at-an-intersectio...
| kragen wrote:
| If you always turn right at every intersection, you will
| just go around and around the same block. Which way you
| should turn depends on where you want to go.
| cgriswald wrote:
| You didn't originally say anything about always turning
| right at every intersection and neither did the GP. I had
| the same two thoughts as GP when I read your analogy.
| diffeomorphism wrote:
| No. That is what roundabouts, curved roads etc are for.
| Left turns are generally more problematic due to crossing
| incoming traffic etc.. Hence planning avoids them for
| good reason and there are much more right turns.
| dmoy wrote:
| Right (ha), but that's kinda how one can approach passive
| voice too?
|
| If you never use passive voice, you will be unable to
| emphasize the object of the sentence in cases where it
| might actually be necessary, and end up requiring more
| words to get the same effect.
|
| If you never make left turns, you end up having to go
| past one block and make three right turns.
|
| So even though best practices might be to avoid passive
| language for various reasons, sometimes it is cleaner.
| And even though best practices are to avoid left turns
| (for efficiency, safety, etc), sometimes it's worth it to
| just take the left turn. So even UPS trucks will make
| left turns, just not nearly as often.
| kragen wrote:
| Well, I do think you should make right turns more often
| than left turns (in countries that drive on the right),
| and you should use the active voice more often than the
| passive voice, say three to ten times more often. But
| that's very different from the advice to only ever make
| right turns or to only ever use the passive voice.
|
| Which way you should turn depends on where you are trying
| to go; which voice you should use depends on what you are
| trying to say, who your audience is, what you want to
| emphasize, and so on.
| plemer wrote:
| I cherish your pedantry. If not here, where?
| Veen wrote:
| I always like to share this when the passive voice comes
| up:
|
| https://youtube.com/playlist?list=PLNRhI4Cc_QmsihIjUtqro3
| uBk...
| kragen wrote:
| Pullum is fantastic, thanks! I didn't know he'd recorded
| video lectures on this topic.
| exe34 wrote:
| My favourite: "a decision was made to...".
|
| It means "I decided to do this, but I don't have the
| balls to admit it."
| IggleSniggle wrote:
| That's funny, I always thought that meant, "my superior
| told me I had to do this obviously stupid thing but I'm
| not going to say my superior was the one who decided this
| obviously stupid thing." Only occasionally, that is said
| in a tongue-and-cheek way to refer directly to the
| speaker as the "superior in charge of the decision."
| dylan604 wrote:
| That reads like several comments I've left in code when
| I've been told to do something very obviously dumb, but
| did not want to get tagged with the "why was it done this
| way?" by the next person reading the code
| SoftTalker wrote:
| You're both right; I've seen it used both ways.
| horsawlarway wrote:
| That's funny because I read this entirely differently
| (somewhat dependent on context)
|
| "A decision was made to..." is often code for "The
| current author didn't agree with [the decision that was
| made] but it was outside their ability to influence"
|
| Often because they were overruled by a superior, or
| outvoted by peers.
| Boldened15 wrote:
| Usually the passive voice is used at work to emphasize
| that it was a team/consensus decision, adjacent to the
| blameless incident management culture. It's not important
| that one engineer or PM pushed it, but that ultimately
| the decision was aligned on and people should be aware.
|
| Although arguably it would be clearer with the active
| voice and which specific teams / level of leadership
| aligned on it, usually in the active voice people just
| use the royal "we" instead for this purpose which doesn't
| add any clarity.
|
| Alternatively sometimes I don't know exactly who made the
| decision, I just learned it from an old commit summary.
| So in that case too it's just important that some people
| at some time made the decision, hopefully got the right
| approvals, and here we are.
| coliveira wrote:
| Many times this is exactly what we want: to emphasize the
| action instead of who is doing it. It turns out that
| technical writing is one of the main areas where we want
| this! So I have always hated this kind of blanket
| elimination of passive voice.
| insane_dreamer wrote:
| The subject can also be the feature itself.
| active/passive:
|
| - The Manage User menu item changes a user's status from
| active to inactive.
|
| - A user's status is changed from active to inactive
| using the Manage User menu item.
| normie3000 wrote:
| Object-orientated vs subject-orientated?
| trealira wrote:
| The passive voice just switches the roles so that the
| patient is the subject and the agent is the object (e.g.
| in "The ball was kicked by John," the ball is still the
| patient despite being the subject). It's just that with
| English word order, it also switches the places of the
| things in the sentence.
|
| In languages with more flexible word order, you could
| just switch the two without passive voice. You could just
| say the equivalent of "The ball kicked John," with it
| being clear somehow that the ball is the grammatical
| object and John the subject, without needing to use the
| passive voice at all.
| kragen wrote:
| While that's true, many of those languages with more
| flexible word order, such as classical Greek and
| classical Latin, _also_ have the passive voice. Classical
| Greek even has a third voice called the "middle voice".
| trealira wrote:
| You're right. Those languages have morphological passive
| voice conjugations for their verbs. That, combined with
| their flexible word order, offers expressivity.
|
| I was just pointing out that English, due to its strict
| word order, is more reliant on the passive voice to
| change word order than less inflexibly-ordered languages.
|
| To borrow from a sentence I used in an earlier comment,
| here's a fragment of Spanish.
|
| "...solo porque te impresiono un espectaculo de magia
| barato."
|
| The equivalent English would be "...just because you were
| impressed by a cheap magic show."
|
| The English sentence has to use the passive voice to put
| the verb "impress" at the beginning of that phrase,
| whereas you still use the active voice in Spanish, just
| with the word order putting the verb first.
| kragen wrote:
| I agree. The OVS order in that Spanish clause is
| unremarkable, though SOV is perhaps more common "un
| espectaculo de magia barato te impresiono". Up to the
| 19th century I think SVO or VOS would have been
| acceptable but now sound archaic: "un espectaculo de
| magia barato impresionote", "impresionote un espectaculo
| de magia barato", and as far as I know OSV and VSO are
| completely forbidden: _" te un espectaculo de magia
| barato impresiono", _"impresiono un espectaculo de magia
| barato te".
|
| You can play tricks to come close to OSV and VSO for
| purposes of emphasis: "A vos un espectaculo de magia
| barato te impresiono", "Te impresiono un espectaculo de
| magia barato a vos," but the "te" is still obligatory.
| And you can do something similar in informal or poetic
| English: "Just because, _you_ , a cheap magic show
| impressed _you_. " But the passive offers more
| flexibility. I posted some other English examples
| yesterday in
| https://news.ycombinator.com/item?id=44493065.
|
| But Spanish's inflectional structure is very much reduced
| from classical Latin, with a corresponding reduction in
| word-order flexibility. I think any of the six
| permutations discussed above would be perfectly valid in
| classical Latin, although my Latin is very weak indeed,
| so I wouldn't swear to it.
| plemer wrote:
| Then we agree.
| dylan604 wrote:
| > Passive - wordy, awkward: The proposal was approved by
| the manager.
|
| Oh the horror. There are 2 additional words "was" and
| "by". The weight of those two tiny little words is so so
| cumbersome I can't believe anyone would ever use those
| words. WTF??? wordy? awkward?
| badlibrarian wrote:
| 29% overhead (two of seven words) adds up.
| dylan604 wrote:
| great, someone can do math, but it is not awkward nor
| wordy.
| badlibrarian wrote:
| It's wordy to a high school teacher. Like using "nor"
| incorrectly it will cause some people's brows to furrow.
| Always best to be aware of the rules you choose to break.
| suzzer99 wrote:
| I reduced my manuscript by 2,000 words with Grammarly. At
| 500 pages, anything I could do to trim it down is a big
| plus.
| ipaddr wrote:
| #2 Is the most pleasant form. The proposal being approved
| is the most important. #1 Tries to make the manager
| approving more important then the approval.
| bityard wrote:
| In addition to the points already made, passive voice is
| painfully boring to read. And it's literally everywhere in
| technical documentation, unfortunately.
| kragen wrote:
| I don't think it's boring. It's easy to come up with
| examples of the passive voice that aren't boring at all.
| It's everywhere in the best writing up to the 19th
| century. You just don't notice it when it's used well
| unless you're looking for it.
|
| Consider:
|
| > _Now we are engaged in a great civil war, testing
| whether that nation, or any nation so conceived and so
| dedicated, can long endure._
|
| This would not be improved by rewriting it as something
| like:
|
| > _Now the Confederacy has engaged us in a great civil
| war, testing whether that nation, or any nation whose
| founders conceived and dedicated it thus, can long
| endure._
|
| This is not just longer but also weaker, because what if
| someone else is so conceiving and so dedicating the
| nation? The people who are still alive, for example, or
| the soldiers who just fought and died? The passive voice
| cleanly covers all these possibilities, rather than just
| committing the writer to a particular choice of who it is
| whose conception and dedication matters.
|
| Moreover, and unexpectedly, the passive voice "we are
| engaged" takes responsibility for the struggle, while the
| active-voice rephrasing "the Confederacy has engaged us"
| seeks to evade responsibility, blaming the Rebs. While
| this might be factually more correct, it is unbefitting
| of a commander-in-chief attempting to rally popular
| support for victory.
|
| (Plausibly the active-voice version is easier to
| understand, though, especially if your English is not
| very good, so the audience does matter.)
|
| Or, consider this quote from Ecclesiastes:
|
| > _For there is no remembrance of the wise more than of
| the fool for ever; seeing that which now is in the days
| to come shall all be forgotten._
|
| You could rewrite it to eliminate the passive voice, but
| it's much worse:
|
| > _For there is no remembrance of the wise more than of
| the fool for ever; seeing that everyone shall forget all
| which now is in the days to come._
|
| This forces you to present the ideas in the wrong order,
| instead of leaving "forgotten" for the resounding final
| as in the KJV version. And the explicit agent "everyone"
| adds nothing to the sentence; it was already obvious.
| joshmarinacci wrote:
| I think what you were saying is that it depends entirely
| on the _type_ of writing you're doing and who your
| audience is.
| kragen wrote:
| I think those are important considerations, but it
| depends even more on what you are attempting to express
| in the sentence in question. There's plenty of active-
| voice phrasing in the Gettysburg Address and Ecclesiastes
| that would not be improved by rewriting it in the passive
| voice.
| thaumasiotes wrote:
| > Consider:
|
| >> _Now we are engaged in a great civil war, testing
| whether that nation, or any nation so conceived and so
| dedicated, can long endure._
|
| > This would not be improved by rewriting it as something
| like:
|
| >> Now the Confederacy has engaged us in a great civil
| war [...]
|
| It's technically possible to parse "we are engaged" as a
| verb in the passive voice.
|
| But it's an error to think that's how you should parse
| it. That clause is using the active verb _be_ , not the
| passive verb _engage_ ; it's fully parallel to "Now we
| are happy".
| kragen wrote:
| You could be right.
| DonHopkins wrote:
| Mistakes were made in the documentation.
| umanwizard wrote:
| You used passive voice in the very first sentence of your
| comment.
|
| Rewriting "the points already made" to "the points people
| have already made" would not have improved it.
| brookst wrote:
| Thats not passive voice. Passive voice is painfully
| boring to read is active. The preamble can be read like
| "however", and is unnecessary; what a former editor of
| mine called "throat-clearing words".
| umanwizard wrote:
| Why isn't it passive voice?
| kragen wrote:
| Yes, the verb "is" in "Passive voice is painfully boring
| to read" is in the active voice, not the passive voice.
| But umanwizard was not saying that "is" was in the
| passive voice. Rather, they were saying that the past
| participle "made", in the phrase "the points already
| made", is a passive-voice use of the verb "make".
|
| I don't know enough about English grammar to know whether
| this is correct, but it's not the assertion you took
| issue with.
|
| Why am I not sure it's correct? If I say, "In addition to
| the blood so red," I am quite sure that "red" is not in
| the passive voice, because it's not even a verb. It's an
| adjective. Past participles are commonly used as
| adjectives in English in contexts that are unambiguously
| not passive-voice verbs; for example, in "Vito is a made
| man now," the past participle "made" is being used as an
| attributive adjective. And this is structurally different
| from the attributive-verb examples of "truly verbal
| adjectives" in
| https://en.wikipedia.org/wiki/Attributive_verb#English,
| such as "The cat sitting on the fence is mine," and "The
| actor given the prize is not my favorite;" we could
| grammatically say "Vito is a man made whole now". That
| page calls the "made man" use of participles "deverbal
| adjectives", a term I don't think I've ever heard before:
|
| > _Deverbal adjectives often have the same form as (and
| similar meaning to) the participles, but behave
| grammatically purely as adjectives -- they do not take
| objects, for example, as a verb might. For example: (...)
| Interested parties should apply to the office._
|
| So, is "made" in "the points already made" really in
| passive voice as it would be in "the points that are
| already made", is it deverbal as it would be in "the
| already-made points" despite its positioning after the
| noun (occasionally valid for adjectives, as in "the blood
| so red"), or is it something else? I don't know. The
| smoothness of the transition to "the points already made
| by those numbskulls" (clearly passive voice) suggests
| that it _is_ a passive-voice verb, but I 'm not sure.
|
| In sibling comment
| https://news.ycombinator.com/item?id=44493969 jcranmer
| says it's something called a "bare passive", but I'm not
| sure.
|
| It's certainly a hilarious thing to put in a comment
| deploring the passive voice, at least.
| thaumasiotes wrote:
| > But umanwizard was not saying that "is" was in the
| passive voice. Rather, they were saying that the past
| participle "made", in the phrase "the points already
| made", is a passive-voice use of the verb "make".
|
| > I don't know enough about English grammar to know
| whether this is correct, but it's not the assertion you
| took issue with.
|
| The most natural interpretation is indeed that the
| participle _made_ is being used as a full participle and
| not as a zero-derived adjective. For example, you could
| give it a really strong verbal sense by saying "the
| points already made at length [...]" or "the points made
| so many times [...]".
|
| > So, is "made" in "the points already made" really in
| passive voice as it would be in "the points that are
| already made"
|
| Though I wouldn't say the same thing there; if you say
| "the points that _are_ already made ", that pretty much
| has to be an adjective. If you want it to be a passive
| verb, go with "the points that have already been made".
|
| Anyway, I would be really surprised if die-hard
| thoughtless style prescriptivists thought that the advice
| "don't use the passive voice" was meant to apply to
| participles. It's a quibble that you don't care about and
| they don't care about or understand. You're never going
| to get anywhere with someone by telling them they mean
| something they know perfectly well they don't mean.
| kragen wrote:
| You say:
|
| > _Anyway, I would be really surprised if die-hard
| thoughtless style prescriptivists thought that the advice
| "don't use the passive voice" was meant to apply to
| participles._
|
| Presumably you mean phrases including participles, not
| participles by themselves. But
| https://languagelog.ldc.upenn.edu/nll/?p=2922 "The
| passive in English" says:
|
| > _The relevance of participles is that a passive clause
| always has its verb in a participial form._
|
| So, what are you saying they _do_ think it was meant to
| apply to, if _every_ passive clause _always_ includes a
| participle? I 'm confused.
|
| With respect to:
|
| > _Though I wouldn 't say the same thing there; if you
| say "the points that are already made", that pretty much
| has to be an adjective. If you want it to be a passive
| verb, go with "the points that have already been made"._
|
| the passive-clause examples given in Pullum's blog post I
| linked above include "Each graduate student is given a
| laptop," which sounds structurally identical to your
| example (except that an indirect object is present,
| showing that it cannot be an adjective) and clarifies:
|
| > _The verb_ was _doesn 't really add any meaning, but it
| enables the whole thing to be put into the preterite
| tense so that the event can be asserted to have occurred
| in the past. Changing_ was _to_ is _would put the clause
| into the present tense, and replacing it by_ will be _or_
| is _going to be would permit reference to future time;
| but the passive VP_ damaged by storms _would stay the
| same in each case. (Notice, the participle_ damaged
| _does_ not _itself make any past time reference, despite
| the name "past participle".)_
|
| So it sounds like your grammatical analysis is explicitly
| contradicting Pullum's, which probably means you're
| wrong, but I'm not sure I understand it.
| thaumasiotes wrote:
| > But https://languagelog.ldc.upenn.edu/nll/?p=2922 "The
| passive in English" says:
|
| >> _The relevance of participles is that a passive clause
| always has its verb in a participial form._
|
| > So, what are you saying they do think it was meant to
| apply to, if _every_ passive clause _always_ includes a
| participle? I 'm confused.
|
| OK, you're confused.
|
| In the general case, an English verb has five forms+:
| "plain form" [go], "preterite form" [went], "present
| third-person singular form" [goes], "-ing form" [going],
| and "-en form" [gone].
|
| The last two of those are participial forms.
|
| It is true that a passive clause always has its verb in a
| participial form. We can be even more specific than that:
| the verb is always in -en form. This is true without
| exception because passive markers occur last in the
| sequence of auxiliary verbs that might modify a primary
| verb, and therefore always directly control the form of
| the primary verb.
|
| It is not true that a passive clause always includes a
| participle, except in the sense of the name we give to
| the form of the verb. -ing and -en are "participial
| forms" because the verb takes one of those forms when it
| is a participle. But it can also take them for other
| reasons.
|
| > the passive-clause examples given in Pullum's blog post
| I linked above include "Each graduate student is given a
| laptop," which sounds structurally identical to your
| example
|
| Sure. If you wanted to put the present passive third-
| person plural form of _make_ in that sentence, that form+
| would be _are made_. The sentence would have all the same
| words in the same order.
|
| But that would make no semantic sense. For a point to
| _be_ "already made", as opposed to _having "already"
| been_ "made", you need to interpret _made_ as an
| adjective, describing the state in which the point
| currently exists. The temporal structure of "each
| graduate student is given a laptop" differs from that of
| "in addition to the points that are already made" in a
| way that allows the present noncontinuous form of the
| verb. I don't think that works for "the points that are
| already made"; if I try to interpret that as a passive
| verb in the present tense, I get a strong sense that the
| sentence is malformed.
|
| + You might notice that these two uses of the word _form_
| conflict with each other. The fact that _form_ is used in
| both of these ways is why I 'm annoyed at your comment
| conflating "participle" with "participial form".
| "Participle" is appropriate when you're talking about
| inflecting a verb according to how you want to use it in
| a sentence; it is a concern with the language's grammar.
| "Participial form" is appropriate when you're talking
| about the actual tokens that can appear in a sentence,
| with no regard to what they might mean or how they might
| be used; it is a concern with what you might think of as
| the language's "anatomy".
| kragen wrote:
| It sounds like you're using the word "participle" in a
| way that conflicts with not just Pullum's explanation but
| also, for example,
| https://en.wikipedia.org/wiki/Participle, which draws no
| distinction between "participle" and "participial form",
| specifically claiming that the past participle and
| present participle _are_ participal forms.
|
| When you say "We can be even more specific than that: the
| verb is always in -en form," you're also contradicting
| Pullum when he points out that the present participle is
| also used to form certain passive clauses. An example of
| this is also given in the Wikipedia article, "I saw John
| eating his dinner."
|
| So, while I certainly agree that I'm confused, I think
| you're mistaken. I'm not sure exactly how, because I'm
| not sure exactly what you're trying to express, but I'm
| sure that sufficient study of sources like those will
| make it clear to you.
| jcranmer wrote:
| "the points already made" is what is known as the "bare
| passive", and yes, it is the passive voice. You can see
| e.g. https://languagelog.ldc.upenn.edu/nll/?p=2922 for a
| more thorough description of the passive voice.
|
| As I said elsewhere, one of the problems with the passive
| voice is that people are so bad at spotting it that they
| can at best only recognize it in its worst form, and
| assume that the forms that are less horrible somehow
| can't be the passive voice.
| kragen wrote:
| I'm not sure this is a "bare passive" like the beginning
| of "The day's work [being] done, they made their way back
| to the farmhouse," one of the bare-passive examples at
| your link. An analogous construction would be, "The
| points already [being] made, I ceased harassing the
| ignorant". But in "In addition to the points already
| made" this case "the point already made" is not a clause;
| it's a noun phrase, the object of the preposition "to".
| Its head is "points", and I believe that "made" is
| modifying that head.
|
| Can you insert an elided copula into it without changing
| the meaning and grammatical structure? I'm not sure. I
| don't think so. I think "In addition to the points
| already being made" means something different: the object
| of the preposition "to" is now "being", and we are going
| to discuss things in addition to that state of affairs,
| perhaps other things that have happened to the points
| (being sharpened, perhaps, or being discarded), not
| things in addition to the points.
| ModernMech wrote:
| "In addition to the points that have already been made"
| kragen wrote:
| I agree that that has the same meaning, but I think it
| may have a different grammatical structure, with an
| entire subordinate clause that was missing from the
| original. Since the voice of a verb is a grammatical
| rather than semantic question, this seems relevant; "in
| addition to the points people have already made" is also
| (probably) semantically equivalent but unquestionably
| uses the active voice.
| PlunderBunny wrote:
| It has its place. We were told to use passive voice when
| writing scientific document (lab reports, papers etc).
| kragen wrote:
| To be fair, current scientific papers are full of utterly
| terrible writing. If you read scientific papers from a
| century and a half ago, a century ago, half a century
| ago, and today, you'll see a continuous and disastrous
| decline in readability, and I think some of that is
| driven by pressure to strictly follow genre writing
| conventions. One of those conventions is using the
| passive voice even when the active voice would be better.
| modriano wrote:
| Are we talking about survivorship bias or are you
| comparing comparably important levels of papers?
| kragen wrote:
| It's hard to correct completely for survivorship bias,
| but it still holds very clearly for very prominent recent
| papers and for other papers that happen to be in the same
| journal issue as the old paper I went there looking for.
| lazyasciiart wrote:
| You could improve this comment by rewriting it in the
| active voice, like this: "I am painfully bored by reading
| passive voice".
| kragen wrote:
| "Is painfully boring" is not the passive voice. I suggest
| reading https://languagelog.ldc.upenn.edu/nll/?p=2922.
| arscan wrote:
| There was a time when Microsoft Word would treat the
| passive voice in your writing with the same level of
| severity as spelling errors or major grammatical mistakes.
| Drove me absolutely nuts in high school.
| PlunderBunny wrote:
| Eventually, a feature was added (see what I did there?)
| that allowed the type of document to be specified, and
| setting that to 'scientific paper' allowed passive voice
| to be written without being flagged as an error.
| Xorakios wrote:
| had to giggle because Microsoft hadn't yet been founded
| when I was in high school!
| jcranmer wrote:
| There's nothing wrong with the passive voice.
|
| The problem is that many people have only a poor ability to
| recognize the passive voice in the first place. This
| results in the examples being clunky, wordy messes that are
| bad because they're, well, clunky and wordy, and not
| because they're passive--indeed, you've often got only a
| fifty-fifty chance of the example passive voice actually
| being passive in the first place.
|
| I'll point out that the commenter you're replying to used
| the passive voice, as did the one they responded to, and I
| suspect that such uses went unnoticed. Hell, I just rewrote
| the previous sentence to use the passive voice, and I
| wonder how many people think recognized that in the first
| place let alone think it worse for being so written.
| suzzer99 wrote:
| Active is generally more concise and engages the reader
| more. Of course there are exceptions, like everything.
|
| Internet posts have a very different style standard than
| a book.
| thaumasiotes wrote:
| > Hell, I just rewrote the previous sentence to use the
| passive voice
|
| Well, sort of. You used the passive voice, but you didn't
| use it on any finite verbs, placing your example well
| outside the scope of the normal "don't use the passive
| voice" advice.
| kragen wrote:
| What would it mean to use the passive voice on a finite
| verb?
| thaumasiotes wrote:
| It would mean that somewhere in your sentence there's a
| clause headed by a passive verb. A finite verb is one
| that heads a clause.
|
| This terminology is where we get the name of the
| "infinitive" form from, by the way.
|
| As a rule of thumb, the nonfinite forms of a verb are its
| infinitives and participles. jcranmer used a passive
| participle, but all of his clauses are active.
| _Unnoticed_ doesn 't have a clause around it.
|
| (He might have thought that _go unnoticed_ is a passive
| form, perhaps of the verb _notice_ (?), in which case
| that would just be an error.)
| hathawsh wrote:
| Here is a simple summary of the common voices/moods in
| technical writing:
|
| - Active: The user presses the Enter key.
|
| - Passive: The Enter key is to be pressed.
|
| - Imperative (aka command): Press the Enter key.
|
| The imperative mood is concise and doesn't dance around
| questions about who's doing what. The reader is expected to
| do it.
| KineticLensman wrote:
| Passive can be disastrous when used in contractual
| situations if the agent who should be responsible for an
| action isn't identified. E.g. "X will be done". I was once
| burnt by a contract that in some places left it unclear
| whether the customer or the contractor was responsible for
| particular tasks. Active voice that identifies the agent is
| less ambiguous
| kragen wrote:
| This is an excellent point, and one I haven't seen raised
| before.
| AdrianB1 wrote:
| Passive is too human. We need robot-styles communications,
| next step is to send json.
| Tainnor wrote:
| absolutely nothing, except that some pencil-pushers with
| poor understanding of grammar somehow dislike it even
| though they often can't even identify it correctly.
|
| Language log has been writing about this for so long it's
| not even funny:
| https://languagelog.ldc.upenn.edu/nll/?cat=54
|
| Although the best place to start is probably here:
| https://languagelog.ldc.upenn.edu/nll/?p=2922
| croes wrote:
| And that's how everything gets flattened to same
| style/voice/etc.
|
| That's like getting rid of all languages and accents and
| switch to the same language
| andrewljohnson wrote:
| The same could be said for books about writing, like
| Williams or Strunk and White. The trick is to not apply
| what you learn indiscriminately.
| bryanlarsen wrote:
| Refusing 2/3rds of grammarly's suggestions flattens
| everything to the same style/voice?
| scubbo wrote:
| No - that was implicitly in response to the sentence:
|
| > The problem is executives want to completely remove
| humans from the loop, which almost universally leads to
| disastrous results.
| kragen wrote:
| I suspect that the disastrous results being envisioned
| are somewhat more severe than not being able to tell who
| wrote which memo. I understood the author to be
| suggesting things more like bankruptcy, global warfare,
| and extermination camps. But it's admittedly ambiguous.
| bryanlarsen wrote:
| Criticisms are almost always read by the reader as
| criticisms of the OP's actions. If you're agreeing with
| somebody as you appear to be here, you should probably
| make that more explicit.
| BeFlatXIII wrote:
| The Esperanto utopia we were denied.
| exe34 wrote:
| I will never use grammarly, not matter how good they get.
| They've interrupted too many videos for me to let it pass.
| dataflow wrote:
| Hasn't Microsoft Word has style checkers for things like
| passive voice for decades?
| adgjlsfhk1 wrote:
| yes, but now they work
| jll29 wrote:
| > The problem is executives want to completely remove humans
| from the loop, which almost universally leads to disastrous
| results
|
| Thanks for your words of wisdom, which touch on a very
| important other point I want to raise: often, we (i.e.,
| developers, researchers) construct a technology that would be
| helpful and "net benign" if deployed as a tool for humans to
| use, instead of deploying it in order to replace humans. But
| then along comes a greedy business manager who reckons
| recklessly that using said technology not as a tool, but in
| full automation mode, results will be 5% worse, but save 15%
| of staff costs; and they decide that that is a fantastic
| trade-off for the company - yet employees may lose and
| customers may lose.
|
| The big problem is that developers/researchers lose control
| of what they develop, usually once the project is completed
| if they ever had control in the first place. What can we do?
| Perhaps write open source licenses that are less liberal?
| kragen wrote:
| You're trying to put out a forest fire with an eyedropper.
|
| Stock your underground bunkers with enough food and water
| for the rest of your life and work hard to persuade the AI
| that you're not a threat. If possible, upload your
| consciousness to a starwisp and accelerate it out of the
| Solar System as close to lightspeed as you can possibly get
| it.
|
| Those measures might work. (Or they might be impossible, or
| insufficient.) Changing your license won't.
| antonvs wrote:
| Alternatively, persuade the AI that you are all-powerful
| and that it should fear and worship you. Probably a more
| achievable approach, and there's precedent for it.
| kragen wrote:
| That only works on the AIs that aren't a real threat
| anyway, and I don't think it helps with the social harm
| done by greedy business managers with less powerful AIs.
| In fact, it might worsen it.
| Bendy wrote:
| That didn't work out for God, we still killed him.
| mistersquid wrote:
| > Alternatively, persuade the AI that you are all-
| powerful and that it should fear and worship you.
|
| I understand this is a bit deeper into one of the _joke_
| threads, but maybe there's something here?
|
| There is a distinction to be made between artificial
| intelligence and artificial consciousness. Where AI can
| be measured, we cannot yet measure consciousness despite
| that many humans could lay plausible claim to possessing
| consciousness (being conscious).
|
| If AI is trained to revere or value consciousness while
| simultaneously being unable to verify it possesses
| consciousness (is conscious), would AI be in a position
| to value consciousness in (human) beings who attest to
| being conscious?
| posnet wrote:
| This one right here Mr. Basilisk
| sjs382 wrote:
| > Grammarly is great for sniffing out extra words and
| passive voice. But it doesn't get writing for humorous
| effect, context, deliberate repetition, etc.
|
| > But then along comes a greedy business manager who
| reckons recklessly
|
| Thanks for this. :)
| kragen wrote:
| I recklessly reckon I will go through the gateless gate
| to hear the sound of one hand clapping.
| csinode wrote:
| The problem here is societal, not technological. An end
| state where people do less work than they do today but
| society is more productive is desirable, and we shouldn't
| be trying to force companies/governments/etc to employ
| people to do an unnecessary job.
|
| The problem is that people who are laid off often
| experience significant life disruption. And people who work
| in a field that is largely or entirely replaced by
| technology often experience permanent disruption.
|
| However, there's no reason it has to be this way - the fact
| people having their jobs replace by technology are
| completely screwed over is a result of the society we have
| all created together, it's not a rule of nature.
| qotgalaxy wrote:
| So it's simple: don't do anything at all about the
| technology that is the impetus for these horrible
| disruptions, just completely rebuild our entire society
| instead.
| selcuka wrote:
| > However, there's no reason it has to be this way - the
| fact people having their jobs replace by technology are
| completely screwed over is a result of the society we
| have all created together, it's not a rule of nature.
|
| I agree. We need a radical change (some version of
| universal basic income comes to mind) that would allow
| people to safely change careers if their profession is no
| longer relevant.
| b3ing wrote:
| No way that will ever happen when we have a party that
| thinks Medicare, Medicaid and social security is
| unnecessary for the poor or middle class. But you better
| believe all our representatives have that covered for
| themselves while pretending to serve us (they only serve
| those that bribe/lobby them)
| dotancohen wrote:
| > the fact people having their jobs replace by technology
| are completely screwed over is a result of the society we
| have all created together, it's not a rule of nature.
|
| How did the handloom weavers and spinners handle the rise
| of the machines?
| throwaway9153 wrote:
| > How did the handloom weavers and spinners handle the
| rise of the machines?
|
| In the past, new jobs appeared that the workers could
| migrate to.
|
| Today, it seems that AI may replace jobs much quicker
| than before and it's not clear to me which new jobs will
| be "invented" to balance the loss.
|
| Optimists will say that we have always managed to invent
| new types of work fast enough to reduce the impact to
| society, but in my opinion it is unlikely to happen this
| time. Unless the politicians figure out a way to keep the
| unemployment content (basic income etc.),
|
| I fear we may end up in a dystopia within our lifetimes.
| I may be wrong and we could end up in a post scarcity
| (star trek) world, but if the current ambitions of the
| top 1% is an indicator, it won't happen unless the
| politicians create a better tax system to compensate the
| loss of jobs. I doubt they will give up wealth and
| influence voluntarily.
| tree-hugger wrote:
| > In the past, new jobs appeared that the workers could
| migrate to.
|
| There was no happy and smooth transition that you seem to
| allude to. The Luddite movement was in direct response to
| this: people were dying over this. Factory owners fired
| or massively reduced wages of workers, replacing many
| with child workers in precarious and dangerous
| conditions. In response, the workers smashed the machines
| that were being used to eliminate their jobs and prevent
| them from feeding themselves and their families (_not_
| the machines that were used to make their jobs easier).
| avereveard wrote:
| I think if we zoom out of the tech and into a bit more of
| economic the risk I see is that the incumbent hold a lot
| of advantages and also control the means of production
| due secondary factors like gpu scarcity.
|
| If we want to draw some parallel this may trigger a
| robber baron kind of outcome more than an industrial
| revolution.
|
| The existence of workable open weight models tips me more
| toward the optimistic outcome
|
| Butthere's trillions at stake now and that must not be
| discounted it's the kind of wealth accumulation that can
| easily trigger a war. (And if you thinkit isn't you can
| look at the oil wars in the 90s and other more recent
| resources war bring fought in Europe today.
|
| Expect "gpu gap" talks sooner that later, and notice
| there's a few global power with no horse to race.
| kragen wrote:
| The CPU-gap and GPU-gap talks started in 02015 and never
| ended before the rise of strategic AI:
| https://www.pcworld.com/article/426879/us-blocks-intel-
| from-...
| Piskvorrr wrote:
| Attempting to unionize. _Then_ the factory owners hired
| thugs to decapitate the movement.
|
| Oh wait, that's not the disneyfied technooptimistic
| version of Luddites? Sorry.
| shafyy wrote:
| > _The problem here is societal, not technological._
|
| I disagree. I think it's both. Yes, we need good
| frameworks and incentivizes on a economic/political
| level. But also, saying that it's not a tech problem is
| the same as saying "guns don't kill people". The truth
| is, if there was no AI tech developed, we would not need
| to regulate it so that greed does not take over. Same
| with guns.
| visarga wrote:
| Oh the web was full of slop long before LLMs arrived.
| Nothing new. If anything, AI slop is higher quality than
| was SEO crap. And of course we can't uninvent AI just
| like we can't unborn a human.
| Sophira wrote:
| It depends on the metric you use.
|
| Yes, AI text could be considered higher quality than
| traditional SEO, but at the same time, it's also very
| much not, because it always _sounds_ like it might be
| authoritative, but you could be reading something
| hallucinated.
|
| In the end, the text was still only ever made to get
| visitors to websites, not to provide accurate
| information.
| jsjohnst wrote:
| > it's also very much not, because it always sounds like
| it might be authoritative, but you could be reading
| something hallucinated
|
| I keep hearing this repeated over and over as if it's a
| unique problem for AI. This is DEFINITELY true of human
| generated content too.
| tpmoney wrote:
| > it's also very much not, because it always sounds like
| it might be authoritative, but you could be reading
| something hallucinated.
|
| People telling lies on the internet is an old enough and
| well known enough issue that it's appeared in children's
| TV shows. One need only dive down the rabbit hole of 9/11
| "truthers" to see how much completely made up bullshit is
| published online as absolute fact with authoritative
| certainty. AI is the hot new thing and gets all the
| headlines, but Scottish Wikipedia was a problem long
| before AI and long after society largely settled its mind
| about how reliable Wikipedia is.
| jsjohnst wrote:
| > The truth is, if there was no AI tech developed, we
| would not need to regulate it so that greed does not take
| over.
|
| Same could be said for the Internet as we know it too.
| Literally replace AI with Internet above and it reads
| equally true. Some would argue (me included some days) we
| are worse off as a society ~30 years later. That's also a
| legitimate case that can be made it was a huge benefit to
| society too. Will the same be said of AI in 2042?
| montagg wrote:
| I think you're describing the principle/agent problem that
| people have wrestled with forever. Oppenheimer comes to
| mind.
|
| You make something, but because you don't own it--others
| caused and directed the effort--you don't control it. But
| the people who control things can't make things.
|
| Should only the people who can make things decide how they
| are used though? I think that's also folly. What about the
| rest of society affected by those things?
|
| It's ultimately a societal decision-making problem: who has
| power, and why, and how does the use of power affect who
| has power (accountability).
| chii wrote:
| > The big problem is that developers/researchers lose
| control
|
| if these developers/researchers are being paid by someone
| else, why should that same someone else be giving up the
| control that they paid for?
|
| If these developers/researchers are paying the research
| themselves (e.g., a startup of their own founding), then
| why would they ever lose control, unless they sell it?
| wmeredith wrote:
| This is a good point. FAANG or whatever you want to call
| it now has spent billions hovering up a couple
| generations' best minds who willing sold their intellect
| to make endless engagement loops.
| Imustaskforhelp wrote:
| The problem of those greedy business managers you speak of
| is that, they don't care how the company does 10 year down
| the line and I almost feel as if everybody is just doing
| things which work short term ignoring the long term
| consequences.
|
| As the comment above said that we need a human in the loop
| for better results, Well firstly it also depends on human
| to human.
|
| A senior can be way more productive in the loop than a
| junior.
|
| So Everybody has just stopped hiring juniors because they
| cost money and they will deal with the AI almost-slop
| later/ someone else will deal with it.
|
| Now the current seniors will one day retire but we won't
| have a new generation of seniors because nobody is giving
| juniors a chance or that's what I've heard about the job
| market being brutal.
| neochief wrote:
| What book did you write?
| eru wrote:
| > The problem is executives want to completely remove humans
| from the loop, which almost universally leads to disastrous
| results.
|
| That's how you get economics of scale.
|
| Google couldn't have a human in the loop to review every page
| of search results before handing them out in response to
| queries.
| oblio wrote:
| Only some things scale like that. Google's insistence to
| use the same model everywhere has gained them a deserved
| reputation as having atrocious support.
| xigoi wrote:
| In the case of a search engine, the human in the loop is
| the user selecting which result to click.
| pbhjpbhj wrote:
| Sure they could. We just want it to be otherwise.
|
| What benefit might human review have? Maybe they could make
| sure the SERP list entries actually have the keywords
| you're looking for. Even better, they could make sure the
| prices in the shopping section are correct! Maybe even make
| sure they relate to the product you actually searched
| for... I might actually pay money for that.
| visarga wrote:
| Yes, we have the context - our unique lived experience, and
| are ultimately accountable for our actions. LLMs have no
| skin. They have no desires, and cannot be punished in any
| way. No matter how smart they get, we are providing their
| opportunities to generate value, guidance and iteration, and
| in the end have to live with the outcomes.
| afavour wrote:
| From my perspective that's fascinatingly upside down thinking
| that leads to you asking to lose your job.
|
| AI is going to get the hang of coding to fill in the spaces
| (i.e. the part you're doing) long before it's able to
| intelligently design an API. Correct API design requires a lot
| of contextual information and forward planning for things that
| don't exist today.
|
| Right now it's throwing spaghetti at the wall and you're
| drawing around it.
| kragen wrote:
| Maybe. So far it seems to be a lot better at creative idea
| generation than at writing correct code, though apparently
| these "agentic" modes can often get close enough after enough
| iteration. (I haven't tried things like Cursor yet.)
|
| I agree that it's also not currently capable of judging those
| creative ideas, so I have to do that.
| bbarnett wrote:
| This sort of discourse really grinds my gears. The framing
| of it, the conceptualization.
|
| It's not creative at all, any more than taking the sum of
| text on a topic, and throwing a dart at it. It's a mild,
| short step beyond a weighted random, and certainly not
| capable of any real creativity.
|
| Myriads of HN enthusiasts often chime in here "Are humans
| any more creative" and other blather. Well, that's a
| whataboutism, and doesn't detract from the fact that
| creative does not exist in the AI sphere.
|
| I agree that you have to judge its output.
|
| Also, sorry for hanging my comment here. Might seem over
| the top, but anytime I see 'creative' and 'AI', I have all
| sorts of dark thoughts. Dark, brooding thoughts with a
| sense of deep foreboding.
| kragen wrote:
| I understand. I share the foreboding, but I try to
| subscribe to the converse of Hume's guillotine.
| Dylan16807 wrote:
| Point taken but if slushing up half of human knowledge
| and picking something to fit into the current context
| isn't creative then humans are rarely creative either.
| LordDragonfang wrote:
| > Well, that's a whataboutism, and doesn't detract from
| the fact that creative does not exist in the AI sphere.
|
| Pointing out that your working definition excludes
| reality isn't whataboutism, it's pointing out an isolated
| demand for rigor.
|
| If you cannot clearly articulate how human creativity
| (the only other type of creativity that exists) is not
| impugned by the definition you're using as evidence that
| creativity "does not exist in the AI sphere", you're not
| arguing from a place of knowledge. Your assertion is just
| as much sophistry as the people who assert it _is_
| creativity. Unlike them, however, you 're having to argue
| against instances where it does appear creative.
|
| For my own two cents, I don't claim to fully understand
| how human creativity emerges, but I _am_ confident that
| all human creative works rest heavily on a foundation of
| the synthesis of author 's previous experiences, both
| personal and of others' creative works - and often more
| heavily the latter. If your justification for a lack of
| creativity is that LLMs are merely synthesizing from
| previous works, then your argument falls flat.
| kragen wrote:
| Agreed.
|
| "Whataboutism" is generally used to describe a more
| specific way of pointing out an isolated demand for rigor
| --specifically, answering an accusation of immoral
| misconduct with an accusation that the accuser is guilty
| of similar immoral misconduct. More broadly,
| "whataboutism" is a term for demands that morality be
| judged justly, by objective standards that apply equally
| to everyone, rather than by especially rigorous standards
| for a certain person or group. As with epistemic rigor,
| the great difficulty with inconsistent standards is that
| we can easily fall into the trap of applying unachievable
| standards to someone or some idea that we don't like.
|
| So it makes some sense to use the term "whataboutism" for
| pointing out an isolated demand for rigor in the
| epistemic space. It's a correct identification of the
| same self-serving cognitive bias that "whataboutism"
| targets in the space of ethical reasoning, just in a
| different sphere.
|
| There's the rhetorical problem that "whataboutism" is a
| _derogatory_ term for demanding that everyone be judged
| by the same standards. Ultimately that makes it
| unpersuasive and even counterproductive, much like
| attacking someone with a racial slur--even if factually
| accurate, as long as the audience isn 't racist, the
| racial slur serves only to tar the speaker with the taint
| of racism, rather than prejudicing the audience against
| its nominal target.
|
| In this specific case, if you concede that humans are no
| more creative than AIs, then it logically follows that
| either AIs are creative to some degree, or humans are not
| creative at all. To maintain the second, you must adopt a
| definition of "creativity" demanding enough to exclude
| all human activity, which is not in keeping with any
| established use of the term; you're using a private
| definition, greatly limiting the usefulness of your
| reasoning to others.
|
| And that is true even if the consequences of AIs being
| creative would be appalling.
| bbarnett wrote:
| I'll play with your tact in this argument, although I
| certain do not agree it is accurate.
|
| You're asserting that creativity is a meld of past
| experience, both personal and the creative output of
| others. Yet this really doesn't jive, as an LLM does not
| "experience" anything. I would argue that raw knowledge
| is not "experience" at all.
|
| We might compare this to the university graduate, head
| full of books and data jammed therein, and yet that
| exceptionally well versed graduate needs "experience" in
| a job for quite some time, before having any use.
|
| The same may be true of learning how to do anything, from
| driving, to riding a bike, or just being in conversations
| with others. Being told, on paper (or as part of your
| baked in, derived "knowledge store") things, means
| absolutely nothing in terms of actually experiencing
| them.
|
| Heck, just try to explain sex to someone before they've
| experienced it. No matter the literature, play, movie or
| act performed in front of them, experience is entirely
| different.
|
| And an AI does not experience the universe, nor is it
| driven by the myriad of human totality, from the mind
| o'lizard, to the flora/fauna in one's gut. There is no
| motive driving it, for example it does not strive to
| mate... something that drives all aspect of mammalian
| behaviour.
|
| So intertwined with the mating urge is human experience,
| that it is often said that all creativity derives from
| it. The sparrow dances, the worm wiggles, and the human
| scores 4 touchdowns in one game, thank you Al.
|
| Comparatively, an LLM does not reason, nor consider, nor
| ponder. It is "born" with full access to all of its
| memory store, has data spewed at it, searches, responds,
| and then dies. It is not capable of learning in any
| stream of consciousness. It does not have memory from one
| birth to the next, unless you feed its own output back at
| it. It can gain no knowledge, except from "context"
| assigned at birth.
|
| An LLM, essentially, understands nothing. It is not
| "considering" a reply. It's all math, top to bottom, all
| probability, taking all the raw info it has an just
| spewing what fits next best.
|
| That's not creative.
|
| Any more than Big Ben's gears and cogs are.
| LordDragonfang wrote:
| Experiences are not materially different from knowledge
| once they are both encoded as memories. They're both just
| encoded in neurons as weights in their network of
| connections. But let's assume there is some ineffable
| difference between firsthand and secondhand experience,
| which fundamentally distinguishes the two in the brain in
| the present.
|
| The core question here, then, is why you are so certain
| that "creativity" requires "experience" beyond knowledge,
| and why knowledge is insufficient? What insight do you
| have into the human mind that top neuroscientists lack
| that grants you this gnosticism on how creativity
| definitely does and does not work?
|
| Because, if you'll permit me to be crude, some of the
| best smut I've read has been by people I'm certain have
| never experienced the act. Their writing has been based
| solely on the writings of others. And yet, knowledge
| alone is more than enough for them to produce evocative
| creative works.
|
| And, to really hammer in a point (please forgive the
| insulting tone):
|
| >It's all math, top to bottom, all probability, taking
| all the raw info it has an just spewing what fits next
| best.
|
| You are just biology, top to bottom, just electrical
| signals, taking all the raw info your nerves get,
| matching patterns and just spewing what fits next best.
|
| Calling LLMs "just math" -- that's not creative, it's
| part of your input that you're predicting fits the likely
| next argument.
|
| You didn't "reason, consider, or ponder" whether I would
| find that argument convincing or be able to immediately
| dismiss it because it holds no weight.
|
| You're simply being a stochastic parrot, repeating the
| phrases you've heard.
|
| ...Etcetera. Again, apologies for the insult. But the
| point I am continually trying to make is that all of the
| arguments everyone tries to make about it not reasoning,
| not thinking, not having creativity -- they all are
| things that can and do apply to almost every human
| person, even intelligent and articulate ones like you or
| I.
|
| When it comes down to it, your fundamental argument is
| that you do not _believe_ that a _machine_ can possibly
| have the exceptional qualities of the human mind, for
| some ineffable reason. It 's all reasoning backwards from
| there. Human creativity _must_ require human-like
| experience, the ability to grow, and a growing context
| cannot _possibly_ suffice, because you 've already
| decided on your conclusion.
|
| (Because, perhaps, it would be too unsettling to admit
| that the alien facsimile of intelligence that we've
| created might have _actual_ intelligence -- so you refuse
| that possibility)
| bbarnett wrote:
| _Experiences are not materially different from knowledge
| once they are both encoded as memories._
|
| The storage medium is not relevant here, and actual
| experience has sensorially laden additions which are not
| currently possible from reading. Memories are laid down
| differently as well.
|
| In terms of what top neuroscientists know or do not know,
| you pulled such out of your back pocket, and threw them
| into onto the floor, perhaps you should describe
| precisely how they negate what I am saying?
|
| Is there consensus? LLM is indeed creative?
|
| What you seem to be missing here, is I am not railing
| against machine intelligence, nor creativity. It is
| merely that an LLM is not it, and will never become it.
| This is no different than an argument over whether to use
| sysvinit or systemd, it is a discussion of technical
| capabilities of a technology.
|
| LLMs may become a backing store, a "library" of sorts for
| any future AGI to use as data, knowledge, an
| exceptionally effective method to provide a wikipedia-ish
| like, non-sql backed data source.
|
| But they provide no means for cognition.
|
| And creativity requires cognition. Creativity is a
| conscious process, for it requires imagination, which is
| an offshoot of a conscious process. Redefining
| "creativity" to exclude the conscious process negates its
| very meaning.
|
| You can say "Wow, this appears to be creative", and it
| may _appear_ to be creative, yet without cognition the
| act is simply not possible. None would dare say that a
| large Goldberg machine, which spits out random answers
| dependent upon air currents was _producing_ creative
| ideas.
|
| Some may say "What a creative creation this machine is!",
| but none would attribute creativity to the _output_ of
| any algorithmic production by that machine, and this is
| what we have here.
|
| Should we derive a method of actual conscious cognition
| in a mind not flesh, so be it. Creativity may occur. But
| as things stand now, a mouse provides more creativity
| than an LLM, the technology is simply not providing the
| underlying requirements. There is no process for
| consciousness.
|
| There are ways to provide for this, and I have pondered
| them (again, I'm validating here that it's not "oh no,
| machines are NOT thinking!!", but instead "LLMs aren't
| that").
|
| One exceptionally rough and barely back-of-napkin
| concept, would be sleep.
|
| No I am not trying to mimic the human mind here, but when
| the concept is examined front to end, the caboose seems
| to be 'sleep'. Right now, the problem is how we bring
| each LLM onto a problem. We simply throw massive context
| at it, and then allow it to proceed.
|
| Instead, we need to have a better context window. Maybe
| we should call this 'short term' memory. An LLM is
| booted, and responds to questions, but has a floating
| context window which never shrinks. Its "context window"
| is not cleared. Perhaps we use a symbolic database, or
| just normal SQL with fluff modz, but we allow this
| ongoing context window to exist and grow.
|
| After a point, this short term memory will grow too large
| to actively swap in/out of memory. Primarily, even RAM
| has bandwidth limits and is a detractor on response speed
| and energy requirements per query.
|
| So -- the LLM "goes to sleep". During that time, the
| backend is converted to LLM, or I suppose in this case a
| small language model. We now have a fuzzy, RRD almost
| conversion of short-term memory to long-term memory, yet
| one which enables some very important things.
|
| That being, an actual capacity to learn from interaction.
|
| The next step is to expand that capability, and the
| capabilities of an LLM with senses. I frankly think the
| best here is real, non-emulated robotic control. Give the
| LLM something to manipulate, as well as senses.
|
| At that point, we should inject agency. A reason to
| exist. With current life, the sole primary reason is
| "reproduce". Everything else has derived from that
| premise. I spoke of the mating urge, we should recreate
| this here.
|
| (Note how non-creative this all actually is, yet it seems
| valid to me... we're just trying to provide what we know
| works as a starting base. It does not mean that we cannot
| expand the creation of conscious minds into other
| methods, once we have a better understanding and
| success.)
|
| There are several other steps here, which are essential.
| The mind must, for example, reload its "long-term memory"
| backing store SLM, and when "sleep" comes, overlay new
| short-term thoughts over long-term. This is another fuzzy
| process, and it would be best to think of it(though
| technically not accurate) as unpacking its SLM,
| overlaying new thoughts, and creating an entirely new
| SLM. As its short-term memory would have output derived
| from the LLM, plus overlaid SLM, its short-term memory
| would be providing output derived from its prior SLM.
|
| So there is a form of continuity here.
|
| So we have:
|
| * A mind which can retain information, and is not merely
| popping into creation with no stored knowledge, and
| killed at each session end
|
| * That same mind has a longer term memory, which allows
| for ongoing concept modification and integration, eg new
| knowledge affecting the "perception" of old knowledge
|
| * That SLM will be "overlaid" on top of its LLM, meaning
| experiences derived during waking moments will provide
| more context to an LLM (that moment when you ride a bike,
| and comprehend how all the literature you read, isn't the
| same as doing? That moment where you link the two? That's
| in the SLM, and the SLM has higher priority)
|
| * A body (simple as it may be), which allows access to
| the environment
|
| * Senses fed into the process
|
| * Agency, as in, "perform to mate", with aspects of
| "perform" being "examine, discover, be impressive" that
| sort of thing
|
| I think, this overlaid SLM, along with actual empirical
| data would provide a more apt method to simulate some
| form of consciousness. It would at least allow a stream
| of consciousness, regardless of whatever debates we might
| have about humans dying when they sleep (which makes no
| sense, as the brain is incredibly active during sleep,
| and constantly monitoring the surroundings for danger).
|
| I'd speak more along the sensory aspects of this, but
| it's actually what I'm working on right now.
|
| But what's key here, is independent data and sensory
| acquisition. I see this is a best-able way to kickstart a
| mind.
| simonw wrote:
| I find it's often way better than API design than I expect.
| It's seen so many examples of existing APIs in its training
| data that it tends to have surprisingly good "judgement" when
| it comes to designing a new one.
|
| Even if your API is for something that's never been done
| before, it can usually still take advantage of its training
| data to suggest a sensible shape once you describe the new
| nouns and verbs to it.
| bryanlarsen wrote:
| I used this to great success just this morning. I told the AI
| to write me some unit tests. It flailed and failed badly at
| that task. But how it failed was instructive, and uncovered a
| bug in the code I wanted to test.
| kragen wrote:
| Haha, that's awesome! Are you going to change the interface?
| What was the bug?
| bryanlarsen wrote:
| It used nonsensical parameters to the API in way that I
| didn't realize was possible (though obvious in hindsight).
| The AI got confused; it didn't think the parameters were
| nonsensical. It also didn't quite use them in the way that
| triggered the error. However it was close enough for me to
| realize that "hey, I never though of that possibility". I
| needed to fix the function to return a proper error
| response for the nonsense.
|
| It also taught me to be more careful about checkpointing my
| work in git before letting an agent go wild on my codebase.
| It left a mess trying to fix its problems.
| kragen wrote:
| Yeah, that's a perfect example of what I'm talking about!
| ChaoPrayaWave wrote:
| In a way, AI's failure can be its own kind of debugger. By
| watching where it stumbles, you sometimes spot flaws you'd
| have missed otherwise.
| momojo wrote:
| A light-weight anecdote:
|
| Many many python image-processing libraries have an `imread()`
| function. I didn't know about this when designing our own
| bespoke image-lib at work, and went with an esoteric
| `image_get()` that I never bothered to refactor.
|
| When I ask ChatGPT for help writing one-off scripts using the
| internal library I often forget to give it more context than
| just `import mylib` at the top, and it almost always defaults
| to `mylib.imread()`.
| kragen wrote:
| That's a perfect example! I wonder if changing it would be an
| improvement? If you can just replace image_get with imread in
| all the callers, maybe it would save your team mental effort
| and/or onboarding time in the future.
| data-ottawa wrote:
| I strongly prefer `image_get/image_read` for clarity, but I
| would just stump in a method called `imread` which is
| functionally the same and hide it from the documentation.
| dimatura wrote:
| I don't know if there's an earlier source, but I'm guessing
| Matlab originally popularized the `imread` name, and that
| OpenCV (along with its python wrapper) took it from there,
| same for scipy. Scikit-image then followed along, presumably.
| bandofthehawk wrote:
| As someone not familiar with these libraries, image_get or
| image_read seems much clearer to me than imread. I'm
| wondering if the convention is worse than your instinct in
| this case. Maybe these AI tools will push us towards
| conventions that aren't always the best design.
| kragen wrote:
| image_get _is_ clearer--unless you 've used Matlab, Octave,
| matplotlib, SciPy, OpenCV, scikit-learn, or other things
| that have copied Matlab's interface. In that case, using
| the established name is clearer.
|
| (Unless, on the gripping hand, your image_get function is
| subtly different from Matlab's imread, for example by not
| returning an array, in which case a different name might be
| better.)
| slowmovintarget wrote:
| Plus one for the _The Mote in God 's Eye_ reference.
| oblio wrote:
| What do libjpeg and family use?
| layer8 wrote:
| HDD -- hallucination-driven development
| codingwagie wrote:
| This works for UX. I give it vague requirements, and it
| implements something i didnt ask for, but is better than i
| would have thought of
| skygazer wrote:
| You're fuzzing the API, unusually, before it's written.
| groestl wrote:
| > and being unintuitive is the only one that this will fix
|
| That's also how I'm approaching it. If all the condensed common
| wisdom poured into the model's parameters says that this is how
| my API is supposed to work to be intuitive, how on earth do I
| think it should work differently? There needs to be a good
| reason (like composability, for example). I break expectations
| otherwise.
| escapecharacter wrote:
| This is similar to an old HCI design technique called Wizard of
| Oz by the way, where a human operator pretends to be the app
| that doesn't exist yet. It's great for discovering new
| features.
|
| https://en.m.wikipedia.org/wiki/Wizard_of_Oz_experiment
| kragen wrote:
| I'd never heard that term! Thank you! I feel like LLMs ought
| to be fantastic at doing that, as well. This is sort of like
| the inverse.
| data-ottawa wrote:
| This was a big problem starting out writing MCP servers for me.
|
| Having an LLM demo your tool, then taking what it does wrong or
| uses incorrectly and adjusting the API works very very well.
| Updating the docs to instruct the LLM on how to use your tool
| does not work well.
| a_e_k wrote:
| I've played with a similar idea for writing technical papers.
| I'll give an LLM my draft and ask it to explain back to me what
| a section means, or otherwise quiz it about things in the
| draft.
|
| I've found that LLMs can be kind of dumb about understanding
| things, and are particularly bad at reading between the lines
| for anything subtle. In this aspect, I find they make good
| proxies for inattentive anonymous reviewers, and so will try to
| revise my text until even the LLM can grasp the key points that
| I'm trying to make.
| kragen wrote:
| That's fantastic! I agree that it's very similar.
|
| In both cases, you might get extra bonus usability if the
| reviewers or the API users actually give your output to the
| same LLM you used to improve the draft. Or maybe a more
| harshly quantized version of the same model, so it makes more
| mistakes.
| djsavvy wrote:
| how do prompt it to make it guess about the API for a library?
| I'm confused how you would structure that in a useful way.
| kragen wrote:
| Often I've started with some example code that invokes part
| of the API, but not all of it. Or in C I can give it the .h
| file, maybe without comments.
|
| Sometimes I can just say, "How do I use the <made-up name>
| API in Python to do <task>?" Unfortunately the safeguards
| against hallucinations in more recent models can make this
| more difficult, because it's more likely to tell me it's
| never heard of it. You can usually coax it into suspension of
| disbelief, but I think the results aren't as good.
| slowmovintarget wrote:
| That's not creativity.
|
| That's closer to simply observing the mean. For an analogy,
| it's like waiting to pave a path until people tread the grass
| in a specific pattern. (Some courtyard designers used to do
| just that. Wait to see where people were walking first.)
|
| Making things easy for Chat GPT means making things close to
| ordinary, average, or mainstream. Not creative, but can still
| be valuable.
| sigbottle wrote:
| Best way to put it. It's very hard to discuss even slightly
| unique concepts with GPT. It just keeps strawmanning ideas
| back to a common consensus without actually understanding the
| deep idea.
|
| On the bright side, a lot of work is just finding the mean
| solution so.
| ldeian wrote:
| > Sometimes it comes up with a better approach than I had
| thought of. Then I change the API so that its code works.
|
| "Sometimes" being a very important qualifier to that statement.
|
| Claude 4 naturally doesn't write code with any kind of long
| term maintenance in-mind, especially if it's trying to make
| things look like what the less experienced developers wrote in
| the same repo.
|
| Please don't assume just because it looks smart that it is.
| That will bite you hard.
|
| Even with well-intentional rules, terrible things happen. It
| took me weeks to see some of it.
| rcthompson wrote:
| In a similar vein, some of my colleagues have been feeding
| their scientific paper methods sections to LLMs and asking them
| to implement the method in code, using the LLM's degree of
| success/failure as a vague indicator of the clarity of the
| method description.
| Cthulhu_ wrote:
| That's a pretty good exercise in writing requirements, with a
| much faster feedback cycle than having developers write it.
| sitkack wrote:
| Explain it Like I am Qwen32
| djtango wrote:
| In essence, a LLM is a crystallisation of a large corpus human
| opinion and you are using that to focus group your API as it is
| representative of a reasonable third party perspective?
| junon wrote:
| Yeah, basically. For example, it's really good at generating
| critical HN comments. Whenever I have a design or an idea I
| formulate it to GPT and ask it to generate a bunch of
| critical HN comments. It usually points out stuff I hadn't
| considered, or at least prepares me to think about and answer
| the tough questions.
| eru wrote:
| > These are ways to harness what neural networks are best at:
| not providing accurate information but making shit up that is
| highly plausible, "hallucination". Creativity, not logic.
|
| This is also similar to which areas TD-Gammon excelled at in
| Backgammon.
|
| Which is all pretty amusing, if you compare it to how people
| usually tended to characterise computers and AI, especially in
| fiction.
| dotancohen wrote:
| > I don't have to spend my time carefully tracking down the
| bugs GPT-4 has cunningly concealed in its code
|
| If anyone is stuck in this situation, give me a holler. My
| Gmail username is the same as my HN username. I've always been
| the one to hunt down my coworkers' bugs, and I think I'm the
| only person on the planet will finds it enjoyable to find
| ChatGPT'S oversights and sometimes seemingly malicious intent.
|
| I'll charge you, don't get me wrong, but I'll save you time,
| money, and frustration. And future bug reports and security
| issues.
| visarga wrote:
| When I see comments like yours I can't help but decry how bad
| was the "stochastic parrots" framing. A parrot does not
| hallucinate a better API.
| Applejinx wrote:
| "Should we really be developing features in response to
| misinformation?"
|
| No, because you'll be held responsible for the misinformation
| being accurate: users will say it is YOUR fault when they learn
| stuff wrong.
| carlosjobim wrote:
| Either the user is a non-paying user and it doesn't matter what
| they think, or the user is a paying customer and you will be
| happy to make and sell them the feature they want.
| Applejinx wrote:
| This is why you will fail.
| carlosjobim wrote:
| Focusing on creating high value for real customers instead
| of chasing people who aren't really interested is a great
| recipe for success. I wouldn't want to do business in any
| different way.
| rorylaitila wrote:
| I've come across something related when building the indexing
| tool for my vintage ad archive using OpenAI vision. No matter how
| I tried to prompt engineer the entity extraction into the defined
| structure I was looking for, OpenAI simply has its own ideas.
| Some of those ideas are actually good! For example it was
| extracting celebrity names, I hadn't thought of that. For other
| things, it would simply not follow my instructions. So I decided
| to just mostly match what it chooses to give me. And I have a
| secondary mapping on my end to get to the final structure.
| colechristensen wrote:
| There are tools for defining structured outputs also called
| grammars which aren't instructions.
|
| Example:
|
| https://llama-cpp-agent.readthedocs.io/en/latest/structured-...
| rorylaitila wrote:
| Thanks, I'll check it out
| felixarba wrote:
| > ChatGPT was outright lying to people. And making us look bad in
| the process, setting false expectations about our service.
|
| I find it interesting that any user would attribute this issue to
| Soundslice. As a user, I would be annoyed that GPT is lying and
| wouldn't think twice about Soundslice looking bad in the process
| romanhn wrote:
| While AI hallucination problems are widely known to the
| technical crowd, that's not really the case with the general
| population. Perhaps that applies to the majority of the user
| base even. I've certainly known folks who place inordinate
| amount of trust in AI output, and I could see them misplacing
| the blame when a "promised" feature doesn't work right.
| carlosjobim wrote:
| The thing is that it doesn't matter. If they're not customers
| it doesn't matter at all what they think. People get false
| ideas all the time of what kind of services a business might
| or might not offer.
| dontlikeyoueith wrote:
| > If they're not customers it doesn't matter at all what
| they think
|
| That kind of thinking is how you never get new customers
| and eventually fail as a business.
| carlosjobim wrote:
| It is the kind of thinking that almost all businesses
| have. You have to focus on the actual products and
| services which you provide and do a good job at it, not
| chase after any and every person with an opinion.
|
| Down voters here on HN seem to live in a egocentric
| fantasy world, where every human being in the outside
| world live to serve them. But the reality is that
| business owners and leaders spend their whole day
| thinking about how to please their customers and their
| potential customers. Not other random people who might be
| misinformed.
| graeme wrote:
| If people _repeatedly_ have a misunderstanding about or
| expectation of your business you need to address it
| though. An llm hallucination is based on widespread norms
| in training data and it is at least worth asking "would
| this be a good idea?"
| smaudet wrote:
| I think the issue here would be that we don't really know
| just how widespread, nor the impact of the issue.
|
| Ok, sure, maybe this feature was worth having?
|
| But if some people start sending bad requests your way
| because they can't or only program poorly, it doesn't
| make sense to potentially degrade the service for your
| successful paying customers...
| carlosjobim wrote:
| An LLM will say that you sell your competitors products
| or that your farm sells freshly harvested strawberries in
| the middle of winter. There are no limits to what kind of
| lies an LLM will invent, and a business owner would be a
| fool to feel responsible for anything an LLM has told
| people about their business or products.
|
| The best LLMs available right in this moment will lie
| without remorse about bus schedules and airplane
| departure times. How in the world are businesses supposed
| to take responsibility for that?
|
| Likewise if I have a neighbour who is a notorious liar
| tell me I can find a piece of equipment in a certain
| hardware store, should I be mad at the store owner when I
| don't find it there, or should I maybe be mad at my
| neighbour - the notorious liar?
| graeme wrote:
| >Likewise if I have a neighbour who is a notorious liar
| tell me I can find a piece of equipment in a certain
| hardware store, should I be mad at the store owner when I
| don't find it there, or should I maybe be mad at my
| neighbour - the notorious liar?
|
| If you are a store own, AND
|
| 1. People repeatedly coming in to your shop asking to buy
| something, AND
|
| 2. It is similar to the kinds of things you sell, from
| the suppliers you usually get supplies from, AND
|
| 3. You don't sell it
|
| Then it sounds like your neighbour the notorious liar is
| doing profitable marketing for your business and sending
| you leads which you could profitably sell to, if you sold
| the item.
|
| If there's a single customer who arrives via
| hallucination, ignore it. If there's a stream of them,
| why would you not serve them if you can profit by doing
| so?
|
| There are obviously instances you'd ignore and you seem
| to be focussing on those rather than what OP was
| obviously talking about, repeat instances of sensible
| ideas
| carlosjobim wrote:
| I guarantee that most businesses have nothing against
| stocking or offering items that people come in and ask
| them for - but it has to be possible also. If you're
| asking for a Polish sausage in a hardware store because
| your neighbour sent you, then they probably don't have
| the licenses to sell food items, and probably their
| profit margin is higher selling hardware than selling
| sausages so they have no reason to start offering them.
|
| There's usually a good reason why a business might not
| offer something that people think they should offer.
| Usually it is that they can't be profitable enough at a
| price point which customers will accept.
| epidemian wrote:
| > You have to focus on the actual products and services
| which you provide and do a good job at it, not chase
| after any and every person with an opinion.
|
| But, this story (and the GP comment) is not talking about
| "any person with an opinion". It's talking about actual
| ChatGPT users. People who've used ChatGPT as a service,
| and got false information from it. Even if they were
| free-tier users (do we even know that?), i think it makes
| sense for them to have some expectations about the
| service working somewhat correctly.
|
| And in the concrete case of these LLM chat services, many
| people do get the impression that the responses they give
| _must_ be correct, because of how deceptively sure and
| authoritative they sound, even when inventing pure BS.
| Sharlin wrote:
| A frighteningly large fraction of non-technical population
| doesn't know that LLMs hallucinate all the time and takes
| everything they say totally uncritically. And AI companies do
| almost nothing to discourage that interpretation, either.
| pphysch wrote:
| The user might go to Soundslice and run into a wall, wasting
| their time, and have a negative opinion of it.
|
| OTOH it's free(?) advertising, as long as that first impression
| isn't too negative.
| jedbrooke wrote:
| slightly off topic: but on the topic of AI coding agents making
| up apis and features that don't exist, I've had good success with
| Q telling it to "check the sources to make sure the apis actually
| exist". sometimes it will even request to read/decompile (java)
| sources, and do grep and find commands to find out what methods
| the api actually contains
| excalibur wrote:
| ChatGPT wasn't wrong, it was early. It always knew you would
| deploy it.
|
| "Would you still have added this feature if ChatGPT hadn't
| bullied you into it?" Absolutely not.
|
| I feel like this resolves several longstanding time travel
| paradox tropes.
| PeterStuer wrote:
| More than once GPT-3.5 'hallucinated' an essential and logical
| function in an API that by all reason should have existed, but
| for whatever reason had not been included (yet).
| iugtmkbdfil834 wrote:
| I wonder if we ever get to the point I remember reading about in
| a novel ( AI initially based on emails ), where human population
| is gently nudged towards individuals that in aggregate benefit AI
| goals.
| linsomniac wrote:
| Sounds like you are referring to book 1 in a series, the book
| called "Avogadro Corp: The Singularity Is Closer than It
| Appears" by William Hertling. I read 3-4 of those books, they
| were entertaining.
| oytis wrote:
| That's the most promising solution to AI hallucinations. If LLM
| output doesn't match the reality, fix the reality
| ecshafer wrote:
| I am currently working on the bug where ChatGPT expects that if
| a ball has been placed on a box, and the box is pushed forward,
| nothing happens to the ball. This one is a doozy.
| oytis wrote:
| Yeah, physics is a bitch. But we can start with history?
| dingnuts wrote:
| [flagged]
| simonw wrote:
| Plenty of people have English as a second language. Having an
| LLM help them rewrite their writing to make it better conform
| to a language they are not fluent in feels entirely appropriate
| to me.
|
| I don't care if they used an LLM provided they put their best
| effort in to confirm that it's clearly communicating the
| message they are intending to communicate.
| kragen wrote:
| Yeah, my wife was just telling me how much Grammarly has
| helped her with improving her English.
| avalys wrote:
| What makes you feel so entitled to tell other people what to
| do?
| kragen wrote:
| Anyone is entitled to make a request--or to ignore one.
| soganess wrote:
| There is a big difference between the above 'request' and,
| say, me politely asking the time of a complete stranger I
| walk by on the street.
|
| Requests containing elements of hostility, shame, or injury
| frequently serve dual purposes: (1) the ostensible aim of
| eliciting an action and (2) the underlying objective of
| inflicting some from of harm (here shame) as a means
| compelling compliance through emotional leverage. Even if
| the respondent doesn't honor the request, the secondary
| purpose still occurs.
| kragen wrote:
| These are good points, but I think they represent a
| somewhat narrow view of the issue. What's happening here
| is that we're discussing among ourselves what kinds of
| actions would be good or bad with respect to AI, just as
| we would with any other social issue, such as urban
| development, immigration, or marital infidelity. You
| could certainly argue that saying "please don't replace
| wetlands with shopping malls" or "please don't immigrate
| to the United States" has "the underlying objective of
| inflicting some from of harm (here shame) as a means [of]
| compelling compliance through emotional leverage."
|
| But it isn't a given that this will be successful; the
| outcome of the resulting conversation may well be that
| shopping malls are, or a particular shopping mall is,
| more desirable than wetlands, in which case the
| ostensible respondent will be _less_ likely to comply
| than they would have been without the conversation. And,
| in this case, it seems that the conversation is strongly
| tending toward favoring the use of things like Grammarly
| rather than opposing it.
|
| So I don't oppose starting such conversations. I think
| it's better to discuss ethical questions like this
| openly, even though sometimes people suffer shame as a
| result.
| pvg wrote:
| Hectoring someone to 'stop doing this' is not 'starting a
| conversation', it's just hectoring.
| kragen wrote:
| A conversation on the topic certainly did ensue; see
| https://news.ycombinator.com/item?id=44492524 and
| https://news.ycombinator.com/item?id=44493015. Perhaps
| you mean to say that this wasn't the intended effect? But
| it was at least a highly predictable effect. Perhaps it
| would have gone better for the flamer if they had made
| the request without flaming not only the author in
| question and simonw.
|
| To me the request in question seems to be in the same
| spirit as "Please don't play your music so loud at
| night", "Please don't look at my sister", or "Please
| don't throw your trash out your car window". In each of
| these cases, there's clearly a conflict between different
| people's desires, probably accompanied with underlying
| disagreements about relevant duties; perhaps one person
| believes the other has a duty to avert their gaze from
| the sister in question to show respect to her chastity,
| while their interlocutor does not subscribe to any such
| duty, believing he is entitled to look at whomever he
| pleases. Or perhaps one person believes the other has a
| duty to carry their trash to a trash can, while the other
| does not.
|
| Given that such a conflict has arisen, how can we resolve
| it? We could merely refrain from trying to influence one
| another's behavior entirely, which is the lowest-effort
| approach, but this clearly leads to deeply suboptimal
| outcomes in many cases; perhaps the cost of turning down
| the stereo or carrying the garbage to a trash can would
| be almost trivial, so doing it to accommodate others'
| preferences results in a net improvement in welfare.
| Alternatively, we could try to exclude people whose
| normative beliefs differ from our own from the spaces
| that most affect us, but it should be obvious that this
| also often causes harms far out of proportion from the
| good that results, such as ethnic cleansing.
|
| All the other approaches to resolving the conflict that I
| can think of--bargaining, mediation, arbitration,
| collective deliberation, etc.--begin unavoidably with
| _stating the unfulfilled desire_. Or, as you put it,
| hectoring someone to 'stop doing this'.
| pvg wrote:
| There's no analogy or wall of text that makes that
| comment unshitty and inviting of conversation. It's not a
| thing one should do on HN because it trashes the place.
| We resolve this by striving to control our own reflexive
| dickishness and downvoting/flagging the egregiously
| dickish comments, which is exactly what happened here.
| kragen wrote:
| I agree that it's a dickish, shitty comment, and
| uninviting of conversation. I don't agree that the reason
| it's dickish is that comments of the form "Please don't
| do such and such" are inherently dickish. I think that
| such comments are uncomfortable but necessary, and
| tabooing discussion of such conflicts does more harm than
| good--in addition to the reasons above, it would ensure
| that only the most disagreeable commenters dare to make
| them.
|
| Undoubtedly, if you devote the minute and a half required
| to read my "wall of text" comment above, you will be
| persuaded by its reasoning.
| pvg wrote:
| I fixed the confusing bit, thanks. I'm not persuaded by
| the reasoning because I don't see how the reasoning is
| relevant - we're talking about a specific dickish comment
| in a specific social place with its specific norms. These
| are so well understood and established the comment got
| flagblasted by users and moderator scolded on top of that
| - effectively the maximum penalty/public shaming an HN
| comment can get. It's not a hypothetical different
| context in which some kind of hypothetical value
| eventually comes from such comments - the bad comment and
| the bad subthread are concretely in front of us.
| kragen wrote:
| I think the non-hypothetical value that came from this
| comment in this case is that it surfaced good reasons for
| writers to use generative AI and showed that many people
| support doing so. I would have liked to see that happen
| in a much more civil fashion, but I don't think it could
| happen at all without some openly stated form of the
| initial objection to writers using generative AI. So I
| think that's the wrong aspect of the initial comment to
| taboo.
|
| My concern is that the flagblasting and moderator-
| scolding, while certainly justified by the comment in
| question, will cause the collateral damage of
| discouraging politer versions of such comments in the
| future. So I think it's worthwhile to affirm that
| criticizing people's behavior to their face is not in
| fact inherently dickish, but rather a much better
| alternative to doing it behind their back, or to finding
| ways to silently exclude them, or people you suspect of
| being like them.
| alwa wrote:
| Does this extend to the heuristic TFA refers to? Where they end
| up (voluntarily or not) referring to what LLMs hallucinate as a
| kind of "normative expectation," then use that to guide their
| own original work and to minimize the degree to which they're
| unintentionally surprising their audience? In this case it
| feels a little icky and demanding because the ASCII tablature
| feature feels itself like an artifact of ChatGPT's limitations.
| But like some of the commenters upthread, I like the idea of
| using it for "if you came into my project cold, how would you
| expect it to work?"
|
| Having wrangled some open-source work that's the kind of genius
| that only its mother could love... there's a place for
| _idiosyncratic_ interface design (UI-wise and API-wise), but
| there's also a whole group of people who are _great_ at that
| design sensibility. That category of people doesn't always
| overlap with people who are great at the underlying
| engineering. Similarly, as academic writing tends to
| demonstrate, people with interesting and important ideas aren't
| always people with a tremendous facility for writing to be
| read.
|
| (And then there are people like me who have neither--I agree
| that you should roll your eyes at anything I ask an LLM to
| squirt out! :)
|
| But GP's technique, like TFA's, sounds to me like something
| closer to that of a person with something meaningful to say,
| who now has a patient close-reader alongside them while they
| hone drafts. It's not like you'd take half of your test
| reader's suggestions, but some of them might be good in a way
| that didn't occur to you in the moment, right?
| tomhow wrote:
| Please don't do this here. If a comment seems unfit for HN,
| please flag it and email us at hn@ycombinator.com so we can
| have a look.
|
| We detached this subthread from
| https://news.ycombinator.com/item?id=44492212 and marked it off
| topic.
| tosh wrote:
| hallucination driven development
| chaboud wrote:
| I had a smaller version of this when coding on a flight (with no
| WiFi! The horror!) over the Pacific. Llama hallucinated array-
| element operations and list-comprehension in C#. I liked the
| shape of the code otherwise, so, since I was using custom
| classes, I just went ahead and implemented both features.
|
| I also went back to just sleeping on those flights and using
| connected models for most of my code generation needs.
| andybak wrote:
| Curious to see the syntax and how it compares to Linq
| chaboud wrote:
| I ended up closer to python, but not totally delighted with
| it (still need to pass in a descriminator function/lambda, so
| it's more structurally verbose). I'd just recommend Linq, but
| I was writing for an old version of Unity coerced through
| IL2CPP (where Linq wasn't great). It was also a chunk of
| semi-hot code (if it was really hot, it wouldn't be sitting
| in C# in Unity), so some of the allocation behaviors of Linq
| behind the scenes wouldn't have been optimal.
|
| What surprised me initially was just how confidently wrong
| Llama was... Now I'm used to confident wrongness from smaller
| models. It's almost like working with real people...
| moomin wrote:
| Is this going to be the new wave of improving AI accuracy? Making
| the incorrect answers correct? I guess it's one way of achieving
| AGI.
| jpadkins wrote:
| Pretty good example of how a super-intelligent AI can control
| human behavior, even if it doesn't "escape" its data center or
| controllers.
|
| If the super-intelligent AI understands human incentives and is
| in control of a very popular service, it can subtly influence
| people to its agenda by using the power of mass usage. Like how a
| search engine can influence a population's view of an issue by
| changing the rankings of news sources that it prefers.
| scinadier wrote:
| Will you use ChatGPT to implement the feature?
| dr_dshiv wrote:
| In addition, we might consider writing the scientific papers
| ChatGPT hallucinates!
| bux93 wrote:
| That's agentic AI, right? Run the LLM in a loop and give it a
| tool to publish to arxiv. If it cites a paper that doesn't
| exist, make it write and upload that one too, recursively.
| Should work for lawyers, too.
| shermantanktop wrote:
| The music notation tool space is balkanized in a variety of ways.
| One of the key splits is between standard music notation and
| tablature, which is used for guitar and a few other instruments.
| People are generally on one side or another, and the notation is
| not even fully compatible - tablature covers information that
| standard notation doesn't, and vice versa. This covers fingering,
| articulations, "step on fuzz pedal now," that sort of thing.
|
| The users are different, the music that is notated is different,
| and for the most part if you are on one side, you don't feel the
| need to cross over. Multiple efforts have been made (MusicXML,
| etc.) to unify these two worlds into a superset of information.
| But the camps are still different.
|
| So what ChatGPT did is actually very interesting. It hallucinated
| a world in which tab readers would want to use Soundslice. But,
| largely, my guess is they probably don't....today. In a future
| world, they might? Especially if Soundslice then enables
| additional features that make tab readers get more out of the
| result.
| adrianh wrote:
| I don't fully understand your comment, but Soundslice has had
| first-class support for tablature for more than 10 years now.
| There's an excellent built-in tab editor, plus importers for
| various formats. It's just the _ASCII tab_ support that 's new.
| kragen wrote:
| I wonder if LLMs will stimulate ASCII formats for more
| things, and whether we should design software in general to
| be more textual in order to work better with LLMs.
| QuercusMax wrote:
| I've had AI create ascii-art (Nethack-style) dungeon
| diagrams when I asked it to write me a D&D adventure. Last
| time I tried it these dungeon diagrams were completely
| nonsensical, but that was a few years ago.
| kragen wrote:
| I think ASCII art in particular is generally a weak point
| for LLMs; maybe they could do better if they used an
| image model. Other ASCII syntaxes using matched { }
| delimiters seem like they would be easier.
| shermantanktop wrote:
| I'm not super familiar with Soundslice. But all the tab users
| I know use guitar pro or maybe ultimate guitar, and none of
| them can read standard notation on its own. Does Soundslice
| have a lot of tab-first users?
| adrianh wrote:
| Yes, Soundslice has a ton of tab-first users. And in fact
| the primary reason I founded the site was to scratch my own
| itch of being able to create tab that's synced with real
| audio recordings. (I'm a guitarist myself.)
| Thorrez wrote:
| When I read the blog post, I thought it was saying that
| Soundslice didn't have any tab support.
|
| The comparison of "we expect this (classical notation
| screenshot) but instead got this (ascii tab screenshot)"
| made me think that the only thing Soundslice supported
| was classical notation.
| adrianh wrote:
| Definitely a subtle distinction there. Soundslice
| supports tab in many formats (MusicXML, Guitar Pro,
| PowerTab, TuxGuitar, PDF/images of published tab, or tab
| notated directly in the Soundslice editor) but didn't
| support ASCII format yet.
| Workaccount2 wrote:
| People forget that while technology grows, society also grows to
| support that.
|
| I already strongly suspect that LLMs are just going to magnify
| the dominance of python as LLMs can remove the most friction from
| its use. Then will come the second order effects where libraries
| are explicitly written to be LLM friendly, further removing
| friction.
|
| LLMs write code best in python -> python gets used more -> python
| gets optimized for LLMs -> LLMs write code best in python
| zamadatix wrote:
| LLMs removing friction from using coding languages would, at
| first glance, seem to erode Python's advantage rather than
| solidify it further. As a specific example LLMs can not only
| spit out HTML+JS+CSS but the user can interact with the output
| directly in browser/"app".
| jjani wrote:
| In a nice world it should be the other way around. LLMs are
| better at producing typed code thanks to the added context and
| diagnostics the types add, while at the same time greatly
| lowering their initial learning barrier.
|
| We don't live in a nice world, so you'll probably end up right.
| johnea wrote:
| What the hell, we elect world leaders based on misinformation,
| why not add s/w features for the same reason?
|
| In our new post truth, anti-realism reality, pounding one's head
| against a brick wall is often instructive in the way the brain
| damage actually produces great results!
| jonathaneunice wrote:
| Paving the folkways!
|
| Figuring out the paths that users (or LLMs) actually want to take
| --not based on your original design or model of what paths they
| should want, but based on the paths that they actually do want
| and do trod down. Aka, meeting demand.
| giancarlostoro wrote:
| Forget prompt engineering, how do you make ChatGPT do this for
| anything you want added to your project that you have no control
| over? Lol
| zzo38computer wrote:
| There are a few things which could be done in the case of a
| situation like that:
|
| 1. I might consider a thing like that like any other feature
| request. If not already added to the feature request tracker, it
| could be done. It might be accepted or rejected, or more
| discussion may be wanted, and/or other changes made, etc, like
| any other feature request.
|
| 2. I might add a FAQ entry to specify that it does not have such
| a feature, and that ChatGPT is wrong. This does not necessarily
| mean that it will not be added in future, if there is a good
| reason to do so. If there is a good reason to not include it,
| this will be mentioned, too. It might also be mentioned other
| programs that can be used instead if this one doesn't work.
|
| Also note that in the article, the second ChatGPT screenshot has
| a note on the bottom saying that ChatGPT can make mistakes
| (which, in this case, it does). Their program might also be made
| to detect ChatGPT screenshots and to display a special error
| message in that case.
| thih9 wrote:
| What made ChatGPT think that this feature is supported? And a
| follow up question - is that the direction SEO is going to take?
| swalsh wrote:
| Id guess the answer is gpt4o is an outdated model that's not as
| anchored in reality as newer models. It's pretty rare for me to
| see sonnet or even o3 just outright tell me plausible but wrong
| things.
| antonvs wrote:
| Hallucinations still occur regularly in all models. It's
| certainly not a solved problem. If you're not seeing them,
| either the kinds of queries you're doing don't tend to elicit
| hallucinations, or you're incorrectly accepting them as real.
|
| The example in the OP is a common one: ask a model how to do
| something with a tool, and if there's no easy way to perform
| that operation they'll commonly make up a plausible answer.
| antonvs wrote:
| > What made ChatGPT think that this feature is supported?
|
| It was a plausible answer, and the core of what these models do
| is generate plausible responses to (or continuations of) the
| prompt they're given. They're not databases or oracles.
|
| With errors like this, if you ask a followup question it'll
| typically agree that the feature isn't supported, because the
| text of that question combined with its training essentially
| prompts it to reach that conclusion.
|
| Re the follow-up question, it's almost certainly the direction
| that advertising in general is going to take.
| poulpy123 wrote:
| Nothing. A LLM doesn't think, it just gives probability to
| words
| thih9 wrote:
| Note that I am replying to the submission and reusing the
| wording from its title.
|
| Also, I'm not suggesting an LLM is actually thinking. We've
| been using "thinking" in a computing context for a long time.
| swalsh wrote:
| Chatbot advertising has to be one of the most powerful forms of
| marketing yet. People are basically all the way through the sales
| pipeline when they land on your page.
| spogbiper wrote:
| makes me wonder how this will be commercialized in the future..
| and i don't like it
| wmeredith wrote:
| It's already being commercialized. There is a burgeoning
| field in the SEO-focused content crowd that is building AIO-
| focused content because it's driving enormous amounts of
| traffic.
| nicbou wrote:
| Right down to the lying salespeople!
| sim7c00 wrote:
| i LOVE this despite feeling for the impacted devs and service.
| love me some good guitar tabs, and honestly id totally beleive
| the chatgpt here hah..
|
| what a wonderful incident / bug report my god.
|
| totally sorry for the trouble and amazing find and fix honestly.
|
| sorry i am more amazed than sorry :D. thanks for sharing this !!
| sim7c00 wrote:
| oh, and yeah. totally the guy who plays guitar 20+ years now
| and cant read musical notation. why? we got tabs for 20+ years.
|
| so i am happy you implemented this, and will now look at using
| your service. thx chatgpt, and you.
| pkilgore wrote:
| Beyond the blog: Going to be an interesting world where these
| kinds of suggestions become paid results and nobody has a hope of
| discovering your competitive service exists. At least in that
| world you'd hope the advertiser actually has the feature already!
| jxjnskkzxxhx wrote:
| So now the machines ask for features and you're the one
| implementing them. How the turns have tabled...
| guluarte wrote:
| The problem with LLMs is that in 99% of cases, they work fine,
| but in 1% of cases, they can be a huge liability, like sending
| people to wrong domains or, worse, phishing domains.
| myflash13 wrote:
| A significant number of new signups at my tiny niche SaaS now
| come from ChatGPT, yet I have no idea what prompts people are
| using to get it to recommend my product. I can't get it to
| recommend my product when trying some obvious prompts on my own,
| on other people's accounts (though it does work on my account
| because it sees my chat history of course).
| wrsh07 wrote:
| Add a prompt for referrals that asks them if they're willing to
| link the discussion that helped them find you!
|
| Some users might share it. ChatGPT has so many users it's
| somewhat mind boggling
| jrochkind1 wrote:
| What this immediately makes me realize is how many people are
| currently trying ot figure out how to intentionally get AI chat
| bots to send people to their site, like ChatGPT was sending
| people to this guy's site. SEO for AI. There will be billions in
| it.
|
| I know nothing about this. I imagine people are already working
| on it, wonder what they've figured out.
|
| (Alternatively, in the future can I pay OpenAI to get ChatGPT to
| be more likely to recommend my product than my competitors?)
| londons_explore wrote:
| To win that game, you have to get your site mentioned on lots
| of organic forums that get ingested in the LLM training data.
|
| So winning AI SEO is not so different than regular SEO.
| latexr wrote:
| You're not thinking far ahead enough. It's just a matter of
| time until LLMs get a system prompt to recommend <whatever
| product is paying that week> when users ask a question near
| that space.
| londons_explore wrote:
| I'm pretty surprised that doesn't exist already TBH.
|
| Would be pretty trivial to have a model which recognises
| product recommendations in the output and inserts branded
| equivalents - or nudges the output towards branded
| equivalents.
| insane_dreamer wrote:
| Along these lines, a useful tool might be a BDD framework like
| Cucumber that instead of relying on written scenarios has an LLM
| try to "use" your UX or API a significant number of times, with
| some randomization, in order to expose user behavior that you (or
| an LLM) wouldn't have thought of when writing unit tests.
| mrcwinn wrote:
| It's worth noting that behind this hallucination there were real
| people with ASCII tabs in need of a solution. If the result is a
| product-led growth channel at some scale, that's a big roadmap
| green light for me!
| zkmon wrote:
| This reminds me how the software integraters or implementers
| worked a couple of decades back. They are IT contractors for
| implementing a popular software product such as IBM MQ or SAP etc
| at a client site and maintaining it. They sometimes incorrectly
| claim that some feature exists, and after finding that it doesn't
| exist, they create a ticket to the software vendor asking for it
| as a patch release.
| burnt-resistor wrote:
| AI is of, for, and by vibe coders who don't care about the
| details.
| jagged-chisel wrote:
| Been using LLMs to code a bit lately. It's decent with
| boilerplate. It's pretty good at working out patterns[1]. It does
| like to ping pong on some edits though - edit this way, no back
| that way, no this way again. I did have one build an entire iOS
| app, it made changes to the UI exactly as I described, and it
| populated sample data for all the different bits and bobs. But it
| did an abysmal job at organizing the bits and bobs. Need running
| time for each of the audio files in a list? Guess we need to add
| a dictionary mapping the audio file ID to length! (For the super
| juniors out there: this piece of data should be attached to
| whatever represents the individual audio file, typically a class
| or struct named 'AudioFile'.)
|
| It really likes to cogitate on code from several versions ago.
| And it often insists repeatedly on edits unrelated to the current
| task.
|
| I feel like I'm spending more time educating the LLM. If I can
| resist the urge to lean on the LLM beyond its capabilities, I
| think I can be productive with it. If I'm going to stop teaching
| the thing, the least it can do is monitor my changes and not try
| to make suggestions from the first draft of code from five days
| ago, alas ...
|
| 1 - e.g. a 500-line text file representing values that will be
| converted to enums, with varying adherence to some naming scheme
| - I start typing, and after correcting the first two, it suggests
| the next few. I accept its suggestions until it makes a mistake
| because the data changed, start manual edits again ... I repeated
| this process for about 30 lines and it successfully learned how I
| wanted the remainder of the file edited.
| colechristensen wrote:
| An LLM is like a group of really productive interns with a
| similar set of limitations.
| jongjong wrote:
| Oh my, people complaining about getting free traffic from
| ChatGPT... While most businesses are worried about all their
| inbound traffic drying up as search engine use declines.
| lofaszvanitt wrote:
| And LLMs started to tell pepl what to do :DDD.
| gortok wrote:
| I think folks have taken the wrong lesson from this.
|
| It's not that they added a new feature because there was demand.
|
| They added a new feature because technology hallucinated a
| feature that didn't exist.
|
| The savior of tech, generative AI, was telling folks a feature
| existed that didn't exist.
|
| That's what the headline is, and in a sane world the folks that
| run ChatGPT would be falling over themselves to be sure it didn't
| happen again, because next time it might not be so benign as it
| was this time.
| lexandstuff wrote:
| Sometimes you just have to deal with the world as it is, not
| how you think it should be.
| gortok wrote:
| Is it your argument that the folks that make generative AI
| applications have nothing to improve from this example?
| rustyminnow wrote:
| If I were someone who ran ChatGPT then yeah, I'd have
| something to improve. But most of us aren't, and can't do
| anything about it - so might as well make lemons into
| lemonade instead of getting hung up on the obvious and
| unchangeable fact that our future is filled with AI
| garbage.
| nomel wrote:
| > in a sane world the folks that run ChatGPT would be falling
| over themselves to be sure it didn't happen again
|
| This would be a world without generative AI available to the
| public, at the moment. Requiring perfection would either mean
| guardrails that would make it useless for most cases, or no LLM
| access until AGI exists, which are both completely irrational,
| since many people are finding practical value in its current
| _imperfect_ state.
|
| The current state of LLM is useful for what it's useful for,
| warnings of hallucinations are present on every official public
| interface, and its limitations are quickly understood with any
| real use.
|
| Nearly everyone in AI research is working on this problem,
| directly or indirectly.
| gortok wrote:
| No one is "requiring perfection", but hallucination is a
| major issue and is in the opposite direction of the "goal" of
| AGI.
|
| If "don't hallucinate" is too much to ask then ethics flew
| out the window long ago.
| nomel wrote:
| > No one is "requiring perfection"
|
| > If "don't hallucinate" is too much to ask then ethics
| flew out the window long ago.
|
| Those sentences aren't compatible.
|
| > but hallucination is a major issue
|
| Again, every official public AI interface has
| warnings/disclaimers for this issue. It's well known. It's
| not some secret. Every AI researcher is directly or
| indirectly working on this.
|
| > is in the opposite direction of the "goal" of AGI
|
| This isn't a logical statement, so it's difficult to
| respond to. Hallucination isn't a direction that's being
| headed towards, it's being actively, with intent and $$$,
| headed _away from_.
| lucianbr wrote:
| > Those sentences aren't compatible.
|
| My web browser isn't perfect, but it does not hallucinate
| inexistent webpages. It sometimes crashes, it sometimes
| renders wrong, it has bugs and errors. It does not invent
| plausible-looking information.
|
| There really is a lot middle gound between perfect and
| "accept anything we give you, no matter how huge the
| problems".
| jeffhuys wrote:
| Different tech, different failure modes.
|
| > it sometimes renders wrong
|
| Is close to equivalent.
| tucnak wrote:
| > It does not invent plausible-looking information.
|
| This is where your analogy is falling apart; of course
| web browsers do not "invent plausible-looking
| information" because they don't invent anything in the
| first place! Web browsers represent a distinct set of
| capabilities, and as you correctly pointed out, these are
| often riddled with bugs and errors. If I was making a
| browser analogy, I would point towards fingerprinting;
| most browsers reveal too much information about any given
| user and system, either via cross-site cookies, GPU
| prints, and whatnot. This is an actual example where
| "ethics flew out the window long ago."
|
| As the adjacent commenter pointed out: different
| software, different failure modes.
| Velorivox wrote:
| > which are both completely irrational
|
| Really!?
|
| [0] https://i.imgur.com/ly5yk9h.png
| tucnak wrote:
| Your screenshot is conveniently omitting the disclaimer
| below: "AI responses may include mistakes. Learn more[1]"
|
| [1]: https://support.google.com/websearch/answer/14901683
| Velorivox wrote:
| It isn't doing anything "conveniently", I was not shown
| the disclaimer (nor anything else, I assume it mostly
| failed to load).
|
| In any case, if you really believe a disclaimer makes it
| okay for Google to display blatant misinformation in a
| first-party capacity, we have little to discuss.
| tucnak wrote:
| https://www.google.com/search?q=is+all+of+oregon+north+of
| +ne...
|
| Show more -> Disclaimer and the feedback buttons are
| shown at the end. If you bothered enough to read the full
| response, you would have seen the disclaimer, but you
| never did, so you haven't. For something to be considered
| "misinformation," in the very least the subject of speech
| has to be asserting its truthfulness--and, indeed, Google
| makes no such claims. The claim they're making is
| precisely that its search result-embedded "[..] responses
| may include mistakes." In this specific case, they are
| not asserting truthfulness.
|
| FWIW, Gemini 2.5 Pro answers the question correctly.
|
| The search hints are clearly a low-compute first
| approximation, which is probably correct for most trivial
| questions which is probably the majority of user queries,
| and it's not surprising that it fails in this specific
| instance. The application doesn't allow for reasoning due
| to scale; even Google cannot afford to run reasoning
| traces on every search question. I concur that there's
| very little to discuss: you seemingly made up your mind
| re: LLM technology, and I doubt you will appreciate the
| breaking-up of your semantics to begin with.
| epidemian wrote:
| > Requiring perfection would either mean guardrails that
| would make it useless for most cases, or no LLM access until
| AGI exists
|
| What?? What does AGI have to do with this? (If this was some
| kind of hyperbolic joke, sorry, i didn't get it.)
|
| But, more importantly, the GP only said that in a sane world,
| the ChatGPT creators should be the ones trying to fix this
| mistake on ChatGPT. After all, it's obviously a mistake on
| ChatGPT's part, right?
|
| That was the main point of the GP post. It was not about
| "requiring perfection" or something like that. So please
| let's not attack a straw man.
| nomel wrote:
| > What does AGI have to do with this?
|
| Their requirement is no hallucinations [1], also stated as
| "be sure it didn't happen again" in the original comment.
| If you define a hallucination as something that wasn't in
| the training data, directly or indirectly (indirectly being
| something like an "obvious" abstract concept), then you've
| placed a _profound_ constraint on the system, requiring
| _determinism_. That requirement _fundamentally_ , by the
| non-deterministic statistics that these run on, means you
| cannot use an LLM, as they exist today. They're not "truth"
| machines - use a database instead.
|
| Saying "I don't know", with determinism is only slightly
| different than saying "I know" with determinism, since it
| _requires_ being fully aware of what you _do_ know, not at
| a fact level, but at a _conceptual_ /abstract level. Once
| you have a system that fully reasons about concepts, is
| self aware of its own knowledge, and can _find_ the
| fundamental "truth" to answer a question with
| _determinism_ , you have something indistinguishable from
| AGI.
|
| Of course, there's a terrible hell that lives between those
| two, in the form of: "Error: Question outside of known
| questions." I think a better alternative to this hell would
| be a breakthrough that allowed "confidence" to be
| quantified. So, accept that hallucinations will exist, but
| _present uncertainty_ to the user.
|
| [1] https://news.ycombinator.com/item?id=44496098
| penteract wrote:
| You have a very strong definition of AGI. "Never being
| wrong" is something that humans fall far short of.
| epidemian wrote:
| > If you define a hallucination as something that wasn't
| in the training data, directly or indirectly (indirectly
| being something like an "obvious" abstract concept), then
| [...]
|
| Ok, sure. But why would you choose to define
| hallucinations in a way that is contrary to common sense
| and the normal understanding of what an AI hallucination
| is?
|
| The common definition of hallucinations is basically:
| when AI makes shit up and presents it as fact. (And the
| more technical definition also basically aligns with
| that.)
|
| No one would say that if the AI takes the data you
| provide in the prompt and can deduce a correct answer for
| that specific data --something that is not directly or
| indirectly present in its training data-- it would be
| hallucinating. In fact that would be an expected thing
| for an intelligent system to do.
|
| It seems to me you're trying to discuss with something
| nobody said. You're making it seem that saying "it's bad
| that LLMs can invent wrong/misleading information like
| this and present it as fact, and that the companies that
| deploy them don't seem to care" is equivalent to "i want
| LLMs to be perfect and have no bugs whatsoever", and then
| discuss about how ridiculous is to state the latter.
| bravesoul2 wrote:
| There was demand for the problem. ChatGPT created demand for
| this solution.
| JimDabell wrote:
| You sound like all the naysayers when Wikipedia was new. Did
| you know anybody can go onto Wikipedia and edit a page to add a
| lie!? How can you possibly trust what you read on there!? Do
| you think Wikipedia should issue groveling apologies every time
| it happens?
|
| Meanwhile, sensible people have concluded that, even though it
| isn't perfect, Wikipedia is still very, very useful - despite
| the possibility of being misled occasionally.
| fzeroracer wrote:
| OK, so how do I edit ChatGPT so it stops lying then?
| JimDabell wrote:
| You have ignored my point.
|
| A technology can be extremely useful despite not being
| perfect. Failure cases can be taken into consideration
| rationally without turning it into a moral panic.
|
| You have no ability to edit Wikipedia to stop it from
| lying. Somebody can come along and re-add the lie a
| millisecond later.
| fzeroracer wrote:
| No, I didn't ignore your point. I invalidated it because
| you're comparing apples to oranges.
|
| And yes, I do have the ability to edit Wikipedia. Anyone
| can edit Wikipedia. I can go on any page I want, right
| now, and make changes to the page. If someone readds a
| lie, then eventually we can hit consensus as other
| editors enter the discussion. Wikipedia's basis is formed
| by consensus and controlled by individuals like you and
| I.
|
| ChatGPT is not. It is controlled by one company; I cannot
| go edit the weights of ChatGPT to prevent it from lying
| about my app or anything else I do. I can only petition
| them to change it and hope that either I have enough
| clout or have a legal basis to do so.
| JimDabell wrote:
| You've changed your position. This is what you originally
| said:
|
| > how do I edit ChatGPT so it stops lying then?
|
| This is what you have changed to:
|
| > And yes, I do have the ability to edit Wikipedia.
|
| You do not have the ability to edit Wikipedia _so it
| stops lying_ , which is the relevant factor here.
| latexr wrote:
| No, that is not accurate. Wikipedia has a number of
| guardrails and such an edit war would be detected and
| investigated. Possibly the page protected and the
| offending IP or account banned.
|
| Wikipedia edits are monitored and vandalism is taken
| seriously, especially on the more important pages.
| latexr wrote:
| > despite the possibility of being misled occasionally.
|
| There is a chasm of difference between being misled
| occasionally (Wikipedia) and frequently (LLMs). I don't think
| you understand how much effort goes on behind the scenes at
| Wikipedia. No, not everyone can edit every Wikipedia page
| willy-nilly. Pages for major political figures often can only
| be edited with an account. IPs like those of iCloud Private
| Relay are banned and can't anonymously edit the most basic of
| pages.
|
| Furthermore, Wikipedia was always honest about what it is
| from the start. They managed expectations, underpromised and
| overdelivered. The bozos releasing LLMs talk about them as if
| they created the embryo of god, and giving money to their
| religion will solve all your problems.
| ChrisMarshallNY wrote:
| That's a riot!
|
| ChatGPT routinely hallucinates API calls. ChatGPT flat-out makes
| it from whole cloth. "Apple Intelligence" creates variants of
| existing API calls, Usually, by adding nonexistent arguments.
|
| Both of them will hallucinate API calls that are frequently added
| by programmers through extensions.
| pentagrama wrote:
| Well, I also learned that the developers of this tool are looking
| at the images their users upload.
| ruperthair wrote:
| Why would they not look at the uploaded images, especially when
| the tool fails to parse them?
| kunzhi wrote:
| Funny this article is trending today because I had a similar
| thought over the weekend - if I'm in Ruby and the LLM
| hallucinates a tool call...why not metaprogram it on the fly and
| then invoke it?
|
| If that's too scary, the failed tool call could trigger another
| AI to go draft up a PR with that proposed tool, since hey, it's
| cheap and might be useful.
| garfij wrote:
| We've done varying forms of this to differing degrees of
| success at work.
|
| Dynamic, on-the-fly generation & execution is definitely
| fascinating to watch in a sandbox, but is far to scary (from a
| compliance/security/sanity perspective) without spending a lot
| more time on guardrails.
|
| We do however take note of hallucinated tool calls and have had
| it suggest an implementation we start with and have several
| such tools in production now.
|
| It's also useful to spin up any completed agents and
| interrogate them about what tools they might have found useful
| during execution (or really any number of other post-process
| questionnaire you can think of).
| kunzhi wrote:
| _> Dynamic, on-the-fly generation & execution is definitely
| fascinating to watch in a sandbox, but is far to scary (from
| a compliance/security/sanity perspective) without spending a
| lot more time on guardrails._
|
| Would love love love to hear more on what you are doing here?
| This seems super fascinating (and scary). :)
| dietr1ch wrote:
| TDD meets LLM-driven API design.
|
| I recall that early on a coworker was saying that ChatGPT
| hallucinated a simpler API than the one we offered, albeit with
| some easy to fix errors and extra assumptions that could've been
| nicer defaults in the API. I'm not sure if this ever got
| implemented though, as he was from a different team.
| jbaber wrote:
| If nothing else, I at least get vindication from hallucinations.
| "Yes, I agree, ChatGPT, that (OpenSSL manpage / ffmpeg flag /
| Python string function) _should_ exist. "
| mehulashah wrote:
| Wow! What if we all did this? What is the closure of the feature
| set that ChatGPT can imagine for your product. Is it one that is
| easy for ChatGPT to use? Is it one that is sound and complete for
| your use cases? Is it the best that you can build had you had
| clear requirements upfront?
| davidmurphy wrote:
| love this
| jivings wrote:
| I'm having the same problem (and had a rant about it on X a few
| weeks ago [1]).
|
| We get ~50% of traffic from ChatGPT now, unfortunately a large
| amount of the features it says we have are made up.
|
| I _really_ don 't want to get into a state of ChatGPT-Driven-
| Development as I imagine that will be never ending!
|
| [1]: https://x.com/JamesIvings/status/1929755402885124154
| petesergeant wrote:
| > We've got a steady stream of new users [and a fun blog post]
|
| Neat
|
| > My feelings on this are conflicted
|
| Doubt
| iachilo wrote:
| Loved this article. If you can adapt to the market (even if the
| AI did that) you can provide your users a greater experience.
| kookamamie wrote:
| You're now officially working for the machine, congrats.
| northisup wrote:
| Is this the first AI hallucinated desire path?
| anovikov wrote:
| I think this is the best way to build features. Build something
| that people want! If people didn't want it ChatGPT won't
| recommend it. You got a free ride on the back of a multibillion
| dollar monster - i can't see what's wrong about that.
| strogonoff wrote:
| Adding a feature because ChatGPT incorrectly thinks it exists is
| essentially design by committee--except this committee is neither
| your users nor shareholders.
|
| On the other hand, adding a feature because you believe it is a
| feature your product should have, a feature that fits your vision
| and strategy, is a pretty sound approach that works regardless of
| what made you think of that feature in the first place.
| alex-moon wrote:
| Here's the thing: I don't think ChatGPT per se was the impetus to
| develop this new feature. The impetus was learning that your
| customers desire it. ChatGPT is operating as the kind of "market
| research" tool here, albeit it in a really unusual, inverted way.
| That said, if someone could develop a market research tool that
| worked this way, i.e. users went to it instead of you have to use
| it to go to users, I can see it making quite a packet.
| copirate wrote:
| They only want ASCII tablature parsing because that's what
| ChatGPT produces. If ChatGPT produced standard music notation,
| users would not care about ASCII tablature. ChatGPT has created
| this "market".
| jaakl wrote:
| ASCII tabulature was not invented by ChatGPT, it is decades
| old thing. It is easier to write with basic computer
| capabilities, and also read for ChatGPT (and humans with no
| formal music education), so it is probably even more
| prominent in the Internet than "standard graphical notation".
| So it quite expected that LLMs have learned a lot of that.
| p0nce wrote:
| We've added formant shifting to Graillon
| https://www.auburnsounds.com/products/Graillon.html largely
| because LLMs thought it already had formant-shifting.
| mbf wrote:
| How fast is that new feature growing? Is it a killer feature?
| sambapa wrote:
| Time travelling AGI confirmed, to the moon!
| mikewarot wrote:
| So this model of ChatGPT obviously has been trained with the July
| 2028 dataset by mistake, including this discussion.
|
| It'll all be fine in a few years. :-;
| jedwards1211 wrote:
| It's the new form of giving into the customer, lol
| amradio1989 wrote:
| The comments are kind of concerning. First, ChatGPT did not
| discover unmet demand in the market. It tried to predict what a
| user would want and hallucinated a feature that could meet that
| demand. Both the demand and the feature were hallucinations. Big
| problem.
|
| The user is not going to understand this. The user may not even
| need that feature at all to accomplish whatever it is they're
| doing. Alternatives may exist. The consequences will be severe if
| companies don't take this seriously.
___________________________________________________________________
(page generated 2025-07-08 23:01 UTC)