[HN Gopher] AI Usage Policy
___________________________________________________________________
AI Usage Policy
Author : mefengl
Score : 458 points
Date : 2026-01-23 09:50 UTC (13 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| cxrpx wrote:
| with limited training data that llm generated code must be
| atrocious
| jakozaur wrote:
| See x thread for rationale:
| https://x.com/mitchellh/status/2014433315261124760?s=46&t=FU...
|
| " Ultimately, I want to see full session transcripts, but we
| don't have enough tool support for that broadly."
|
| I have a side project, git-prompt-story to attach Claude Vode
| session in GitHub git notes. Though it is not that simple to do
| automatic (e.g. i need to redact credentials).
| radarsat1 wrote:
| I've thought about saving my prompts along with project
| development and even done it by hand a few times, but
| eventually I realized I don't really get much value from doing
| so. Are there good reasons to do it?
| fragmede wrote:
| It's not for you. It's so others can see how you arrived to
| the code that was generated. They can learn better prompting
| for themselves from it, and also how you think. They can see
| which cases got considered, or not. All sorts of good stuff
| that would be helpful for reviewing giant PRs.
| Ronsenshi wrote:
| Sounds depressing. First you deal with massive PRs and now
| also these agent prompts. Soon enough there won't be any
| coding at all, it seems. Just doomscrolling through massive
| prompt files and diffs in hopes of understanding what is
| going on.
| verdverm wrote:
| I suspect this future will not play out. Mitchell is
| definitely leaning to one side on this debate.
|
| To me, quality code is quality code no matter how it was
| arrived at. That should be the end of it
| simonw wrote:
| For me it's increasingly _the work_. I spend more time in
| Claude Code going back and forth with the agent than I do in
| my text editor hacking on the code by hand. Those transcripts
| ARE the work I 've been doing. I want to save them in the
| same way that I archive my notes and issues and other
| ephemera around my projects.
|
| My latest attempt at this is
| https://github.com/simonw/claude-code-transcripts which
| produces output like the is: https://gisthost.github.io/?c75b
| f4d827ea4ee3c325625d24c6cd86...
| radarsat1 wrote:
| Right, I get that writing prompts is "the work", but if you
| run them again you don't get the same code. So what's the
| point of keeping them? They are not 'source code' in the
| same sense as a programming language.
| simonw wrote:
| That's why I want the transcript that shows the prompts
| AND the responses. The prompts alone have little value.
| The overall conversation shows me exactly what I did,
| what the agent did and the end result.
| radarsat1 wrote:
| > shows me exactly what I did
|
| I get that, but I guess what I'm asking is, why does it
| matter what you did?
|
| The result is working, documented source code, which
| seems to me to be the important part. What value does
| keeping the prompt have?
|
| I'm not trying to needle, I just don't see it.
| awesan wrote:
| If the AI generated most of the code based on these prompts,
| it's definitely valuable to review the prompts before even
| looking at the code. Especially in the case where
| contributions come from a wide range of devs at different
| experience levels.
|
| At a minimum it will help you to be skeptical at specific
| parts of the diff so you can look at those more closely in
| your review. But it can inform test scenarios etc.
| verdverm wrote:
| Using them for evals at a future date.
|
| I save all of mine, including their environment, and plan to
| use them for iterating on my various system prompts and tool
| instructions.
| optimalsolver wrote:
| >I want to see full session transcripts, but we don't have
| enough tool support for that broadly
|
| I think AI could help with that.
| stevenhuang wrote:
| simow wrote a tool that does this for Claude code
|
| https://simonw.substack.com/p/a-new-way-to-extract-
| detailed-...
| ollien wrote:
| Not sure how I feel about transcripts. Ultimately I do my best
| to make any contributions I make high quality, and that means
| taking time to polish things. Exposing the tangled mess of my
| thought process leading up to that either means I have to
| "polish" that too (whatever that ends up looking like), or put
| myself in a vulnerable position of showing my tangled process
| to get to the end result.
| couchdb_ouchdb wrote:
| You should be able to attach the plan file that you and Claude
| develop in Plan mode before even starting to code. This should
| be the source of truth.
| empath75 wrote:
| On our team, we have discussed attaching claude transcripts to
| jira tickets, not github PRs (though the PRs are attached to
| tickets)
| arjunbajaj wrote:
| I can see this becoming a pretty generally accepted AI usage
| policy. Very balanced.
|
| Covers most of the points I'm sure many of us have experienced
| here while developing with AI. Most importantly, AI generated
| code does not substitute human thinking, testing, and clean
| up/rewrite.
|
| On that last point, whenever I've gotten Codex to generate a
| substantial feature, usually I've had to rewrite a lot of the
| code to make it more compact even if it is correct. Adding
| indirection where it does not make sense is a big issue I've
| noticed LLMs make.
| imiric wrote:
| I agree with you on the policy being balanced.
|
| However:
|
| > AI generated code does not substitute human thinking,
| testing, and clean up/rewrite.
|
| Isn't that the end goal of these tools and companies producing
| them?
|
| According to the marketing[1], the tools are already "smarter
| than people in many ways". If that is the case, what are these
| "ways", and why should we trust a human to do a better job at
| them? If these "ways" keep expanding, which most proponents of
| this technology believe will happen, then the end state is that
| the tools are smarter than people at _everything_ , and we
| shouldn't trust humans to do _anything_.
|
| Now, clearly, we're not there yet, but where the line is drawn
| today is extremely fuzzy, and mostly based on opinion. The
| wildly different narratives around this tech certainly don't
| help.
|
| [1]: https://blog.samaltman.com/the-gentle-singularity
| Terretta wrote:
| Intern generated code does not substitute for tech lead
| thinking, testing, and clean up/rewrite.
| imiric wrote:
| No, the code is generated by a tool that's "smarter than
| people in many ways". So which parts of "thinking, testing,
| and clean up/rewrite" can we trust it with?
| cmsj wrote:
| The marketing is irrelevant. The AIs are not aware of
| what they are doing, or motivated in the ways humans are.
| phanimahesh wrote:
| Very little, until it stops being stupid in many ways. We
| don't need smart, we need tools to not be stupid. An
| unreliable tool is more dangerous and more useless than
| having no tool.
| TeMPOraL wrote:
| Trust is a function of responsibility, not of smarts.
|
| You may hire a genius developer that's better than you at
| everything, and you still won't trust them blindly with
| work _you_ are responsible for. In fact, the smarter they
| are than you, the _less_ trusting you can afford to be.
| nicoburns wrote:
| > Isn't that the end goal of these tools and companies
| producing them?
|
| It seems to be the goal. But they seem very far away from
| achieving that goal.
|
| One thing you probably account for is that most of the
| proponents of these technologies are trying to sell you
| something. Doesn't mean that there is no value to these
| tools, but the wild claims about the capabilities of the
| tools are just that.
| OvbiousError wrote:
| This is such a good write-up and something I'm struggling
| with very hard. Does quality of code in the traditional sense
| even matter anymore if e.g. CC can work with said code
| anyway. I haven't had imposter's in a long time, but it's
| spiking hard now. Whenever i read or write code I feel like
| I'm an incompetent dev doing obsolete things.
| fzaninotto wrote:
| I agree this could be a template that services like GitHub
| should propose, the same way as they suggest contributing and
| code of conduct templates.
| dawnerd wrote:
| I think I'm going to use it as a guide for our own internal AI
| guideline. We hire a lot of contractors and the amount of just
| awful code we get is really taking a toll and slowing site
| buildouts.
| ottah wrote:
| Everything except the first provision is reasonable. IMO it's
| none of your damn business how I wrote the code, only that I
| understand it, and am responsible for it.
|
| It's one of those provisions that seem reasonable, but really
| have no justification. It's an attempt to allow something,
| while extracting a cost. If I am responsible for my code, and
| am considered the author in the PR, than you as the recipient
| don't have a greater interest to know than my own personal
| preference not to disclose. There's never been any other
| requirement to disclose anything of this nature before. We
| don't require engineers to attest to the operating system or
| the licensing of the tools they use, so materially outside your
| own purant interests, how does it matter?
| alansaber wrote:
| "Pull requests created by AI must have been fully verified with
| human use." should always be a bare minimum requirement.
| Ntrails wrote:
| > "Pull requests [] must have been fully verified with human
| use."
|
| I would expect this is entirely uncontroversial and the AI
| qualifier redundant.
| Retr0id wrote:
| If you have good tests, certain types of change can be merged
| without manual testing. One problem specific to AI is that it
| has a tendency to game/bypass/nerf/disable tests, as opposed
| to actually making the code do the correct thing.
| verdverm wrote:
| that's why we have CI like GHA that runs the tests on PRs /
| commits
| bwat49 wrote:
| I would hope that actually testing the changes is done
| regardless of whether or not AI is used
| njhnjhnjh wrote:
| AI is so smart these days that I typically just ask Claude to
| verify the code for me.
|
| This sort of request may have made sense in the old days but as
| the quality of generated code rapidly increases, so does the
| necessity of human intervention decrease.
| Applejinx wrote:
| If you care so little, why are you even prompting at all?
| Surely you can leave it to its own devices without troubling
| it with your wishes? It seems like the farther you go down
| this path, the more likely it is that it'll have something
| better to do.
| thunderfork wrote:
| If you're going to put something on someone else's desk,
| you're going to have to own it.
|
| If you don't check it yourself, then you're going to own
| whatever your tooling misses, and also own the amount of
| others' time you waste through what the project has decided
| to categorize as negligence, which will make you look worse
| than if you simply made an honest mistake.
| toraway wrote:
| I literally just scrolled past a thread discussing the
| psychology of shamelessness and undeserved self-confidence
| that creates all this drive-by AI pull request slop wasting
| everyone else's time and see this comment...
| maxnevermind wrote:
| > verified with human use
|
| Quality of that verification matters, people who might use AI
| tend to cut corners. This does not completely solve problem
| with AI slop imo and solution quality. You ask Claude Code to
| go and implement a new feature in a complex code base, it will,
| the code might even work, but implementation might have subtle
| issues and might be missing the broader vision of the repo.
| verdverm wrote:
| > people who might use AI tend to cut corners
|
| People do this all the time too, and is one source for the
| phrase "tech debt"
|
| It's also a biased statement. I use Ai and I cut fewer
| corners now because the Ai can spam out that boring stuff for
| me
| vegabook wrote:
| Ultimately what's happening here is AI is undermining trust in
| remote contributions, and in new code. If you don't know somebody
| personally, and know how they work, the trust barrier is getting
| higher. I personally am already ultra vigilant for any github
| repo that is not already well established, and am even concerned
| about existing projects' code quality into the future. Not
| against AI per se (which I use), but it's just going to get
| harder to fight the slop.
| epolanski wrote:
| Honestly I don't care how people come with the code they create,
| but I hold them responsible for what they try to merge.
|
| I work in a team of 5 great professionals, there hasn't been a
| single instance since Copilot launched in 2022 that anybody, in
| any single modification did not take full responsibility for
| what's been committed.
|
| I know we all use it, to different extent and usage, but the
| quality of what's produced hasn't dipped a single bit, I'd even
| argue it has improved because LLMs can find answers easier in
| complex codebases. We started putting `_vendor` directories with
| our main external dependencies as git subtrees, and it's super
| useful to find information about those directly in their source
| code and tests.
|
| It's really as simple. If your teammates are producing slop,
| that's a human and professional problem and these people should
| be fired. If you use the tool correctly, it can help you a lot
| finding information and connecting dots.
|
| Any person with a brain can clearly see the huge benefit of these
| tools, but also the great danger of not reviewing their output
| line by line and forfeiting the constant work of resolving design
| tensions.
|
| Of course, open source is a different beast. The people
| committing may not be professionals and have no real stakes so
| they get little to lose by producing slop whereas maintainers are
| already stretched in their time and attention.
| embedding-shape wrote:
| > It's really as simple. If you or your teammates are producing
| slop, that's a human and professional problem and these people
| should be fired.
|
| Agree, slop isn't "the tool is so easy to use I can't review
| the code I'm producing", slop is the symptom of "I don't care
| how it's done, as long as it looks correct", and that's been a
| problem before LLMs too, the difference is how quickly you
| reach the "slop" state now, not that you have gate your
| codebase and reject shit code.
|
| As always, most problems in "software programming" isn't about
| software nor programming but everything around it, including
| communication and workflows. If your workflow allows people to
| not be responsible for what they produce, and if allows shitty
| code to get into production, then that's on you and your team,
| not on the tools that the individuals use.
| altmanaltman wrote:
| I mean this policy only applies to outside contributors and not
| the maintainers.
|
| > Ghostty is written with plenty of AI assistance, and many
| maintainers embrace AI tools as a productive tool in their
| workflow. As a project, we welcome AI as a tool!
|
| > Our reason for the strict AI policy is not due to an anti-AI
| stance, but instead due to the number of highly unqualified
| people using AI. It's the people, not the tools, that are the
| problem.
|
| Basically don't write slop and if you want to contribute as an
| outsider, ensure your contribution actually is valid and works.
| kanzure wrote:
| Another project simply paused external contributions entirely:
| https://news.ycombinator.com/item?id=46642012
|
| Another idea is to simply promote the donation of AI credits
| instead of output tokens. It would be better to donate credits,
| not outputs, because people already working on the project would
| be better at prompting and steering AI outputs.
| lagniappe wrote:
| >people already working on the project would be better at
| prompting and steering AI outputs.
|
| In an ideal world sure, but I've seen the entire gamut from
| amateurs making surprising work to experts whose prompt history
| looks like a comedy of errors and gotchas. There's some "skill"
| I can't quite put my finger on when it comes to the way you
| must speak to an LLM vs another dev. There's more monkey-paw
| involved in the LLM process, in the sense that you get what you
| want, but do you want what you'll get?
| CrociDB wrote:
| I recently had to do a similar policy for my TUI feed reader,
| after getting some AI slop spammy PRs:
| https://github.com/CrociDB/bulletty?tab=contributing-ov-file...
|
| The fact that some people will straight up lie after submitting
| you a PR with lots of _that type_ of comment in the middle of the
| code is baffling!
| nutjob2 wrote:
| A factor that people have not considered is that the copyright
| status of AI generated text is not settled law and precedent or
| new law may retroactively change the copyright status of a whole
| project.
|
| Maybe a bit unlikely, but still an issue no one is really
| considering.
|
| There has been a single ruling (I think) that AI generated code
| is uncopyrightable. There has been at least one affirmative fair
| use ruling. Both of these are from the lower courts. I'm still of
| the opinion that generative AI is not fair use because its
| clearly substitutive.
| direwolf20 wrote:
| This only matters if you get sued for copyright violation,
| though.
| christoph-heiss wrote:
| No? Licenses still apply even if you _don't_ get sued?
| latexr wrote:
| Do they? Isn't the application of the license its
| enforcement?
|
| It's illegal to commit fraud or murder, but if you do it
| and suffer no consequences (perhaps you even get pardoned
| by your president), does it matter that it was illegal?
| Laws are as strong as their enforcement.
|
| For a less grim and more explicit example, Apple has a
| policy on the iOS App Store that apps may not use
| notifications to advertise. Yet it happens all the time,
| especially from big players like Uber. Apple themselves
| have done it too. So if you're a bad actor and
| disrespectful to your users, does it matter that the rule
| exists?
| direwolf20 wrote:
| Licenses determine the outcome of copyright lawsuits. If
| there's no copyright lawsuit, nobody looks at the license.
|
| Licenses determine whether a copyright lawsuit is likely to
| happen. Most entities won't sue you if they expect to lose.
| But they are not the only deciding factor. Some entities
| never sue, which means you don't have to follow their
| licenses.
|
| Sometimes they don't sue because they don't think they can
| prove you infringed copyright, even if you did. Even if AI
| is found to be copyright infringement in general, that
| won't mean every output is a copyright infringement of
| every input. Writing C code wouldn't be copyright
| infringement of Harry Potter. The entity suing you would
| still have to prove that you infringed.
| consp wrote:
| At what time in the future does this not become an issue?
| nutjob2 wrote:
| If you're a big enough target, that is inevitable.
|
| You may become a big enough target only when it's too late to
| undo it.
| 101008 wrote:
| I never thought of this, you are right. What happens if, let's
| say, AI generated text/code is "ilegal"? Especially what
| happens with all the companies that have been using it for
| their products? Do they need to rollback? It should be a shit
| show but super interesting to see it unfold...
| Sytten wrote:
| I think the usage is so widespread now that the law will adapt
| to customs. It is untenable now to say code generated is
| uncopyrightable IMO. Maybe copyright as is defined right now is
| not enough, but then the legislation will change it. There is
| enough pressure on them from the business community to do so.
| tpxl wrote:
| I agree with you that generative AI is clearly not fair use.
|
| However, at this point, the economic impact of trying to de
| tangle this mess would be so large, the courts likely won't do
| anything about it. You and I don't get to infringe on
| copyright; Microsoft, Facebook and Google sure do though.
| cess11 wrote:
| Some take that in consideration, I did when I until recently
| was in a CTO role, and I've come across companies that take
| compliance seriously and have decided against such code
| synthesis due to the unclear legal status.
| Version467 wrote:
| The biggest surprise to me with all this low-quality contribution
| spam is how little shame people apparently have. I have a handful
| of open source contributions. All of them are for small-ish
| projects and the complexity of my contributions are in the same
| ball-park as what I work on day-to-day. And even though I am
| relatively confident in my competency as a developer, these
| contributions are probably the most thoroughly tested and
| reviewed pieces of code I have ever written. I just really,
| really don't want to bother someone with low quality "help" who
| graciously offers their time to work on open source stuff.
|
| Other people apparently don't have this feeling at all. Maybe I
| shouldn't have been surprised by this, but I've definitely been
| caught off guard by it.
| Etheryte wrote:
| I worked for a major open-source company for half a decade.
| Everyone thinks their contribution is a gift and you should be
| grateful. To quote Bo Burnham, "you think your dick is a gift,
| I promise it's not".
| latexr wrote:
| > To quote Bo Burnham, "you think your dick is a gift, I
| promise it's not".
|
| For those curious:
|
| https://www.youtube.com/watch?v=llGvsgN17CQ
| Cthulhu_ wrote:
| Sounds like everyone's got some main character syndrome, the
| cure for that is to be a meaningless cog in the enterprise
| wheels for a while. But then I suspect a lot of open source
| contributions are done exactly by those people - they don't
| really matter in their day job, but in open source they can
| Make A Difference.
|
| Of course, the vast majority of OS work is the same cog-in-a-
| machine work, and with low effort AI assisted contributions,
| the non-hero-coding work becomes more prevalent than ever.
| kleiba wrote:
| "Other people" might also just be junior devs - I have seen
| time and again how (over-)confident newbies can be in their
| code. (I remember one case where a student suspected a bug in
| the JVM when some Java code of his caused an error.)
|
| It's not necessarily maliciousness or laziness, it could simply
| be enthusiasm paired with lack of experience.
| xxs wrote:
| have found bugs in native JVM, usually it takes some effort,
| though. Printing the assembly is the easiest one. (I consider
| the bug in java.lang/util/io/etc. code not an interesting
| case)
|
| Memory leaks and issues with the memory allocator are months
| long process to pin on the JVM...
|
| In the early days (bug parade times), the bugs are a lot more
| common, nowadays -- I'd say it'd be an extreme naivete to
| consider JVM the culprit from the get-go.
| JDye wrote:
| Our postgres replication suddenly stopped working and it took
| three of us hours - maybe days - of looking through the
| postgres source before we actually accepted it wasn't us or
| our hosting provider being stupid and submitted a ticket.
|
| I can't imagine the level of laziness or entitlement required
| for a student (or any developer) to blame their tools so
| quickly without conducting a thorough investigation.
| benldrmn wrote:
| Funny, I had a similar experience TAing "Intro to CS" (first
| semester C programming course). The student was certain he
| encountered a compiler bug (pushing back on my assumption
| there was something wrong with their code, since while
| compilers do have bugs, they are probably not in the code
| generation of a nested for loop). After spending a few
| minutes parsing their totally unindented code, the off-by-one
| error revealed itself
| zehaeva wrote:
| Off topic, but I feel like this could be made into a Zen
| Koan from The Codeless Code[0]. You're almost there with
| it!
|
| [0] https://thecodelesscode.com/
| direwolf20 wrote:
| Offer topic, but the Codeless Code isn't Zen Koans. It's
| formatted like Zen Koans, and it's entertaining and
| brings value to the world, but it isn't the same thing.
| DrewADesign wrote:
| To have that shame, you need to know better. If you don't know
| any better, having access to a model that can make code and a
| cursory understanding of the language syntax probably _feels
| like_ knowing how to write good code. Dunning-Krueger strikes
| again.
|
| I'll bet there are probably also people trying to farm accounts
| with plausible histories for things like anonymous supply chain
| attacks.
| arbitrandomuser wrote:
| when it comes to enabling opportunities i dont think it becomes
| a matter of shame for them anymore. A lot of people (especially
| in regions where living is tough and competition is fierce)
| will do anything by hook or crook to get ahead in competition.
| And if github contributions is a metric for getting hired or
| getting noticed then you are going to see it become spammed.
| flexagoon wrote:
| Keep in mind that many people also contribute to big open
| source projects just because they believe it will look good ok
| their CV/GitHub and help them get a job. They don't care about
| helping anyone, they just want to write "contributed to
| Ghostty" in their application.
| nchmy wrote:
| I think this falls under the "have no shame" comment that
| they made
| 0x696C6961 wrote:
| From my experience, it's not about helping anyone or CV
| building. I just ran into a bug or a missing feature that is
| blocking me.
| wccrawford wrote:
| It's worse. Some of them are _required to contribute to an
| existing project of their choice_ for some course they 're
| taking.
| Ronsenshi wrote:
| It's good to regularly see such policies and discussions around
| them to remind me how staggeringly shameless some people could
| be and how many of such people out there. Interacting mostly
| with my peers, friends, acquaintances I tend to forget that
| they don't represent average population and after some time I
| start to assume all people are reasonable and act in good
| faith.
| toyg wrote:
| Yep, this. You can just look at the state of FOSS licensing
| across GitHub to see it in action: licenses are routinely
| stripped or changed to remove the original developers, even
| on trivial items, _even on forked projects_ where the action
| is easily visible, even on licenses that allow for _literally
| everything else_. State "You can do everything except this"
| and loads of people will still actively do it, because they
| have no shame (or because they enjoy breaking someone else's
| rules? Because it gives them a power trip? Who knows).
| latentsea wrote:
| I think of it like people just have crappy prompt
| adherence. It makes more sense that way.
| Ronsenshi wrote:
| With AI at least you can wipe the context and reapply
| system prompt.
| direwolf20 wrote:
| You don't have to obey the copyright of anyone who's not
| willing to sue you for copyright infringement. Business
| moguls know this.
| 6LLvveMx2koXfwn wrote:
| Shamelessness is very definitely in vogue at the moment. It
| will pass, let's hope for more than ruins.
| pixl97 wrote:
| To put this another way, shame is only effective if it's
| coupled with other repercussions with long standing effects.
|
| An example I have of this is from high school where there
| were guys that were utterly shameless in asking girls for
| sex. The thing is it worked for them. Regardless of how many
| people turned them down they got enough of a hit rate it was
| an effective strategy. Simply put there was no other social
| mechanism that provided enough disincentive to stop them.
|
| And to take the position as devil's advocate, why should they
| feel shame? Shame is typically a moral construct of the
| culture you're raised in and what to be ashamed for can vary
| widely.
|
| For example, if your raised in the culture of Abrahamic
| religions it's very likely you're told to be ashamed for
| being gay. Whereas non-religious upbringing is more likely to
| say why the hell would you be ashamed for being gay.
|
| TL:DR, shame is not an effective mechanism on the internet
| because you're dealing with far too many cultures that have
| wildly different views on shame, and any particular viewpoint
| on shame is apt to have millions to billions of people that
| don't believe the same.
| monegator wrote:
| > The biggest surprise to me with all this low-quality
| contribution spam is how little shame people apparently have.
|
| ever had a client second guess you by replying you a screenshot
| from GPT?
|
| ever asked anything in a public group only to have a complete
| moron replying you with a screenshot from GPT or - at least a
| bit of effor there - a copy/paste of the wall of text?
|
| no, people have no shame. they have a need for a little bit of
| (borrowed) self importance and validation.
|
| Which is why i applaud every code of conduct that has public
| ridicule as punishment for wasting everybody's time
| monooso wrote:
| Not OP, but I don't consider these the same thing.
|
| The client in your example isn't a (presumably) professional
| developer, submitting code to a public repository, inviting
| the scrutiny of fellow professionals and potential future
| clients or employers.
| monegator wrote:
| I consider them to be the same attitude. Machine made it /
| Machine said it. It _must_ be right, you _must_ be wrong.
|
| They are sure they know better because they get a yes man
| doing their job for them.
| Aeolun wrote:
| Random people don't do this. Your boss however...
| Sharlin wrote:
| Problem is people seriously believe that whatever GPT tells
| them must be true, because... I don't even know. Just because
| it sounds self-confident and authoritative? Because computers
| are supposed to not make mistakes? Because talking computers
| in science fiction do not make mistakes like that? The fact
| that LLMs ended up having this particular failure mode, out
| of all possible failure modes, is incredibly unfortunate and
| detrimental to the society.
| pousada wrote:
| I think in science fiction it's one of the most common
| themes for the talking computer to be utterly horribly
| wrong, often resulting in complete annihilation of all life
| on earth.
|
| Unless I have been reading very different science fiction I
| think it's definitely not that.
|
| I think it's more the confidence and seeming plausibility
| of LLM answers
| Sharlin wrote:
| Sure, but this failure mode is not _that_. "AI will
| malfunction and doom us all" is pretty far from "AI will
| malfunction by sometimes confabulating stuff".
| oneeyedpigeon wrote:
| People are literally taking Black Mirror storylines and
| trying to manifest them. I think they did a `s/dys/u/`
| and don't know how to undo it...
| direwolf20 wrote:
| They codysld start by trying to dysndo it.
|
| I'm sorry. That was a terrible joke.
| TeMPOraL wrote:
| The stories I read had computers being utterly horribly
| _right_ , which resulted in attempts (sometimes
| successful) at annihilate humanity.
| rsynnott wrote:
| In terms of mass exposure, you're probably talking things
| like Cmdr Data from Star Trek, who was very much on the
| 'infallible' end of the fictional AI spectrum.
| TheSpiceIsLife wrote:
| Data was also famous for getting things embarrassingly
| wrong, particularly when interacting with his human
| colleagues.
| Cthulhu_ wrote:
| I don't remember exactly who said it, but at one point I
| read a good take - people trust these chatbots because
| there's big companies and billions behind them, surely big
| companies test and verify their stuff thoroughly?
|
| But (as someone else described), GPTs and other current-day
| LLMs are probabilistic. But 99% of what they produce seems
| feasible enough.
| pera wrote:
| Last year I had to deal with a contractor who sincerely
| believed that a very popular library had some issue because
| it was erroring when parsing a chatgpt generated json...
| I'm still shocked, this is seriously scary
| TeMPOraL wrote:
| "SELECT isn't broken" isn't a new advice, and it exists
| for a reason.
| anon_anon12 wrote:
| People's trust on LLM imo stems from the lack of awareness
| of AI hallucinating. Hallucination benchmarks are often
| hidden or talked about hastily in marketing videos.
| wpietri wrote:
| I think it's better to say that LLMs only hallucinate.
| All the text they produce is entirely unverified. Humans
| are the ones reading the text and constructing meaning.
| Suzuran wrote:
| My boss says it's because they are backed by trillion
| dollar companies and the companies would face dire legal
| threats if they did not ensure the correctness of AI
| output.
| buggy6257 wrote:
| Your boss sounds hilarious naive to how the world works.
| Suzuran wrote:
| In a lot of ways he is, despite witnessing a lot of how
| the sausage is made directly. Honestly, I think at at
| least half of it is wanting to convince himself that the
| world still functions in ways that make sense to him
| rather than admit that it's mostly grifters grifting all
| the way down.
| nathanaldensr wrote:
| The high-trust Boomer brain cannot comprehend the actual
| low-trust society of grifters in which we live.
| pluralmonad wrote:
| I don't agree with this blanket statement. The internet
| is low trust for lots of reasons, but regular (read
| small, proximal/spatiotemporally constrained) communities
| still exist and are not grifters all the way down.
| Acknowledging that distant strangers are not trustworthy
| in the traditional sense seems reasonable, but is
| categorically different than addressing natural social
| groups (small and local).
| OGEnthusiast wrote:
| Yes, and most young Americans are locked out of those
| small, high-trust suburbs due to high housing prices. So
| instead they get to experience the magic of low-trust
| America first-hand, hence the disconnect between the
| young and the boomers.
| ryandrake wrote:
| Exactly. Sadly, low-trust America has become the default
| where most people live. There are still nice, small-town,
| local shopping, suburban high-trust enclaves here and
| there, but as soon as you go online or deal with a
| business with more than a handful of locations, you're
| back in the low-trust grifting zone.
| direwolf20 wrote:
| The Gervais Principle framework calls this type of person
| a Clueless. They sit in middle management as a buffer
| between the Sociopaths who run the world, and the Losers
| who know the world sucks but would just like to get their
| paycheck and go home. I'm surprised to hear this actually
| play out -- the Gervais Principle doesn't seem very
| empirical.
| TeMPOraL wrote:
| This is a good heuristic, and it's how most things in
| life operate. It's the reason you can just buy food in
| stores without any worry that it might hurt you[0] -
| there's potential for million ${local currency} fines,
| lawsuits, customer loss and jail time serving as strong
| incentive for food manufacturers and vendors to not fuck
| this up. The same is the case with drugs, utilities, car
| safety and other important aspects of life.
|
| So their boss may be naive, but not hilariously so -
| because that is, in fact, how the world works[1]! And as
| a boss, they probably have some understanding of it.
|
| The thing they miss is that AI fundamentally[2] cannot
| provide this kind of "correct" output, and more
| importantly, that the "trillion dollar companies" not
| only don't guarantee that, they actually _explicitly
| inform everyone everywhere, including in the UI, that the
| output may be incorrect_.
|
| So it's mostly failure to pay attention and realize
| they're dealing with an exception to the rule.
|
| --
|
| [0] - Actually hurt you, I'm ignoring all the
| fitness/healthy eating fads and "ultraprocessed food"
| bullshit.
|
| [1] - On a related note, it's also something security
| people often don't get: real world security relies on
| being connected - via contracts and laws and institutions
| - to "men with guns". It's not perfect, but scales
| better.
|
| [2] - Because LLMs are not databases, but - to a first-
| order approximation - little people on a chip!
| miki123211 wrote:
| > [1]
|
| Cybersecurity is also an exception here.
|
| "men with guns" only work for cases where the criminal
| must be in the jurisdiction of the crime for the crime to
| have occurred.
|
| If you rob a bank in London, you must be in London, and
| the British police can catch you. If you rob a bank
| somebody else, the British police doesn't care. If you
| hack a bank in London though, you may very well be in
| North Korea.
| ryandrake wrote:
| > It's the reason you can just buy food in stores without
| any worry that it might hurt you[0] - there's potential
| for million ${local currency} fines, lawsuits, customer
| loss [...]
|
| We are currently facing a political climate trying to
| tear many of these safeguards down. Some people really
| think "caveat emptor" is some kind of natural, efficient,
| ideal way of life.
| saratogacx wrote:
| For this logic I like to point out that every AI service
| has text that says, essentially "AI can be wrong, double
| check your answers". If you had the same disclaimer on
| your food "This food's quality is not assured" would you
| feel comfortable buying it or would you take pause until
| you've built up trust with the seller and manufacturer.
|
| There's so much CYA because there is an A that needs
| C'ing
| johnnyanmac wrote:
| Just doesn't understand the scale of money.
|
| Maybe a million dollar company needs to be compliant. A
| billion dollar company can start to ward off any
| loopholes with lawsuits instead of compliance.
|
| A trillion dollar company will simply change the law and
| fight governments over the law to begin with, rather than
| worrying about compliance.
| breakingcups wrote:
| If only every LLM-shop out there would put disclaimers on
| their page that they hope absolve them of the
| responsibility of correctness, so that your boss could
| make up his own mind... Oh wait.
| rsynnott wrote:
| And just how many rs does your boss think are in
| strawberry?
| tzs wrote:
| Point out to your boss that trillion dollar companies
| have million dollar lawyers making sure their terms of
| service put all responsibility on the user, and if
| someone still tries to sue them they hire $2000/hour
| litigators from top law firms to deal with it.
| tveita wrote:
| I think people's attitude would be better calibrated to
| reality if LLM providers were legally required to call
| their service "a random drunk guy on the subway"
|
| E.g.
|
| "A random drunk guy on the subway suggested that this
| wouldn't be a problem if we were running the latest SOL
| server version" "Huh, I guess that's worth testing"
| TheSpiceIsLife wrote:
| There's a non-zero number of people who would get a
| chuckle out of a browser extension at replaces every
| occurrence of LLM or AI with _a random drunk guy on the
| subway_.
| direwolf20 wrote:
| It could be the same extension that replaces every
| occurrence of _the cloud_ with _my butt_.
| pjc50 wrote:
| Billions of dollars of marketing have been spent to enable
| them to believe that, in order to justify the trillions of
| investment. Why would you invest a trillion dollars in a
| machine that occasionally randomly gave wrong answers?
| johnnyanmac wrote:
| This is probably more of a GAI achievement, but we
| definitely need confidence levels when it comes to making
| queries with factual responses.
|
| But yes, look at the US c.2025-6. As long as the leader
| sounds assertive, some people will eat the blatant lies
| that can be disproven even by the same AI tools they laud.
| positive-spite wrote:
| Didn't happen to me yet.
|
| I'm not looking forward to it...
| TeMPOraL wrote:
| This sounds a bit like the "Asking vs. Guessing culture"
| discussion on the front page yesterday. With the "Guesser"
| being GP who's front-loading extra investigation, debugging
| and maintenance work so the project maintainers don't have to
| do it, and with the "Asker" being the client from your
| example, pasting the submission to ChatGPT and forwarding its
| response.
| slfreference wrote:
| >> In Guess Culture, you avoid putting a request into words
| unless you're pretty sure the answer will be yes. Guess
| Culture depends on a tight net of shared expectations. A
| key skill is putting out delicate feelers. If you do this
| with enough subtlety, you won't even have to make the
| request directly; you'll get an offer. Even then, the offer
| may be genuine or pro forma; it takes yet more skill and
| delicacy to discern whether you should accept.
|
| delicate feelers is like octopus arms
| meindnoch wrote:
| Our CEO chiming in on a technical discussion between
| engineers: by the way, this is what Claude says: *some
| completely made-up bullshit*
| javcasas wrote:
| Hi CEO, thanks for the input. Next time that we have a
| discussion, we will ask Claude instead of discussing with
| who wrote the offending code.
| pixl97 wrote:
| I do want to counter that in the past before AI, the CEO
| would just chime in with some completely off the wall
| bullshit from a consultant.
| ncruces wrote:
| I've also had the opposite.
|
| I raise an issue or PR after carefully reviewing someone
| else's open source code.
|
| They ask Claude to answer me; neither them nor Claude
| understood the issue.
|
| Well, at least it's their repo, they can do whatever.
| Sharlin wrote:
| You just have to go take a look at what people write in social
| media, using their real name and photo, to conclude that no,
| some people have no shame at all.
| weinzierl wrote:
| _" The biggest surprise to me with all this low-quality
| contribution spam is how little shame people apparently have."_
|
| And this is one half of why I think
|
| _" Bad AI drivers will be [..] ridiculed in public."_
|
| isn't a good clause. The other is that ridiculing others, not
| matter what, is just no decent behavior. Putting it as a rule
| in your policy document makes it only worse.
| conartist6 wrote:
| Getting to live by the rules of decency is a privilege now
| denied us. I can accept that but I don't have to like it or
| like the people who would abuse my trust for their personal
| gain.
|
| Tit for tat
| weinzierl wrote:
| It is well supported that TFT with a delayed mirroring
| component and Generous Tit for Tat where you sometimes
| still cooperate after defection are pretty succesful.
|
| What is written in the Ghostty AI policy lacks any nuance
| or generosity. It's more like a Grim Trigger strategy than
| Tit for Tat.
| conartist6 wrote:
| You can't have 1,000,000 abusers and be nuanced and
| generous to all of them all the time. At some point
| either you either choose to knowingly enable the abuse or
| you draw a line in the sand, drop the hammer, send a
| message, whatever you want to call the process of setting
| boundaries in anger. Getting a hammer dropped on them
| isn't going to feel fair to the individuals it falls on,
| but it's also unrealistic to expect that a mob-like group
| can trample with impunity because of the fear of being
| rude or unjust to an individual member of that mob.
|
| It is understanding of these dynamics that lead to us to
| our current system of law: punitive justice, but
| forgiveness through pardons.
| anonymous908213 wrote:
| > The other is that ridiculing others, not matter what, is
| just no decent behavior.
|
| Shaming people for violating valid social norms is absolutely
| decent behaviour. It is the primary mechanism we _have_ to
| establish social norms. When people do bad things that are
| harmful to the rest of society, shaming them is society 's
| first-level corrective response to get them to stop doing bad
| things. If people continue to violate norms, then society's
| higher levels of corrective behaviour can involve things like
| establishing laws and fining or imprisoning people, but you
| don't want to start with that level of response. Although
| putting these LLM spammers in jail _does_ sound awfully
| enticing to me in a petty way, it 's probably not the most
| constructive way to handle the problem.
|
| The fact that shamelessness is taking over in some cultures
| is another problem altogether, and I don't know how you deal
| with that. Certain cultures have completely abdicated the
| ability to influence people's behaviour socially without
| resorting to heavy-handed intervention, and on the internet,
| this becomes everyone in the world's problem. I guess the
| answer is probably cultivation of spaces with strict
| moderation to bar shameless people from participating. The
| problem could be mitigated to some degree if a Github-like
| entity outright banned these people from their platform so
| they could not continue to harass open-source maintainers,
| but there is no platform like that. It unfortunately takes a
| lot of unrewarding work to maintain a curated social
| environment on the internet.
| weinzierl wrote:
| In a functioning society the primary mechanism to deal with
| violation of social norms is (temporary or permanent)
| social exclusion and in consequence the loss of future
| cooperative benefits.
|
| To demand public humiliation doesn't just put you on the
| same level as our medieval ancestors, who responded to
| violations of social norms with the pillory - it's actually
| even worse: the contemporary internet pillory never
| forgets.
| anonymous908213 wrote:
| You think _exile_ is a better first step than shame? That
| 's certainly a take. On the internet, that does manifest
| as my suggested way of dealing with people where shame
| doesn't work, a curated space where offenders are banned
| -- but I would still advocate for attempting lesser
| corrective behaviour first before exclusion. Moreover,
| exclusion only works if you have a means to viably
| exclude people. Shame is something peers can do;
| exclusion requires authority.
|
| Shame is also not the same thing as "public humiliation".
| They are publicly humiliating _themselves_. Pointing out
| that what they publicly chose to do themselves is bad is
| in no way the same as coercing them into being
| humiliated, which is what "public humiliation as a
| medieval punishment" entails. For example, the medieval
| practice of dragging a woman through the streets nude in
| order to humiliate her is indeed abhorrent, but you can
| hardly complain if you march through the streets nude of
| your own volition, against other people's desires, and
| are then publicly shamed for it.
| wpietri wrote:
| No society can function without enforced rules. Most people
| do the pro-social thing most of the time. But for the rest,
| society must create negative experiences that help train
| people to do the right thing.
|
| What negative experience do you think should instead be
| created for people breaking these rules?
| weinzierl wrote:
| Temporary or permanent social exclusion, and consequently
| the loss of future cooperative benefits.
|
| A permanent public internet pillory isn't just useless
| against the worst offenders, who are shameless anyway. It's
| also permanently damaging to those who are still learning
| societal norms.
|
| The Ghostty AI policy lacks any nuance in this regard. No
| consideration for the age or experience of the offender. No
| consideration for how serious the offense actually was.
| ryandrake wrote:
| Thanks to Social Media bubbles, there's no social
| exclusion possible anymore. Shameless people just go
| online find each other and reinforce each others'
| shamelessness. I bet there's a Facebook group for people
| who don't return their shopping carts.
| ionwake wrote:
| TBH Im not sure if this is a "growing up in a good area" vibe.
| But over the last decade or so I have had to slowly learn the
| people around me have no sense of shame. This wasnt their
| fault, but mine. Society has changed and if you don't adapt
| you'll end up confused and abused.
|
| I am not saying one has to lose their shame, but at best,
| understand it.
| pousada wrote:
| Like with all things in life shame is best in moderation.
|
| Too little or too much shame can lead to issue.
|
| Problem is no one tells you what too little or too much
| actually is and there are many different situations where you
| need to figure it out on your own.
|
| So I think sometimes people just get it wrong but ultimately
| everyone tries their best. Truly malicious shameless people
| are extremely rare in my experience.
|
| For the topic at hand I think a lot of these "shameless"
| contributions come from kids
| Cthulhu_ wrote:
| To add, I don't know if this is a cultural, personal, or
| other thing but nowadays even if people get shamed for
| whatever they do, they see it more as a challenge, and it
| makes them rebel even harder against what is perceived to
| be old fashioned or whatever.
|
| Basically teenagers. But it feels like the rebellious
| teenager phase lasts longer nowadays. Zero evidence besides
| vibes and anecdotes, but still.
|
| Or maybe it's me that's getting old?
| ryandrake wrote:
| I feel like there is a growing number of people who just
| can't even recognize or acknowledge shame. It's not even an
| emotion they are capable of or understand.
|
| So many people now respond to "You shouldn't do that..."
| with one or more of:
|
| - But, I'm allowed to.
|
| - But, it's legal.
|
| - But, the rules don't say I can't.
|
| - But, nobody is stopping me.
|
| The shared cultural understanding of _right_ and _wrong_ is
| shrinking. More and more, there 's just _can_ and _can 't_.
| pousada wrote:
| I agree and I think it's a backlash to the 2010's where
| many felt there was too much shame/shaming going on in
| most online spaces.
|
| Fwiw I haven't noticed either phenomenon much irl but
| that might just be my bubble.
| derrida wrote:
| Shame is a good thing it shows one has a conscience and
| positive self regard.
|
| Just like pain is a good thing, it tells you and signals to
| remove your hand from the stove.
| ionwake wrote:
| Yeah but its not helpful if its the new air fryer thats
| burning the hand not the stove, unless you adapt.
| krferriter wrote:
| I've been saying for a couple years now that we need a
| healthy revitalization of shame in society. Sure in the
| past (and present) shaming people has been done for bad
| reasons but shame itself serves an important social
| function and I feel like there has been a collapse in its
| effectiveness, which has been very bad for society. People
| should be made to feel ashamed for certain things they do.
| It should impact them deeply and it should linger with them
| and be reinforced by others around them until they
| successfully make behavior changes. For example I see
| people lie pretty shamelessly and they suffer no lasting
| consequences for it. They should be stained with shame
| until they alter their behavior. People should not let them
| move past it and move on to the next lie.
| wang_li wrote:
| The adaption is going to be that competent, knowledgeable
| people will begin forming informal and formal networks of
| people they know are skilled and intelligent and begin to
| scorn the people who aren't skilled and aren't intelligent.
| They will be less willing to work with people who don't have
| a proven record of competence. This results in greater
| stratification and harder for people who aren't already part
| of the in group to break in.
| warkdarrior wrote:
| > skilled and intelligent [people] begin to scorn the
| people who aren't skilled and aren't intelligent
|
| That has NEVER led to a positive result in the whole of
| human history, especially that the second group is much
| larger than the first.
| Wojtkie wrote:
| It doesn't help that it seems like society has been trending
| to reward individuals with a lack of shame. Fortune favors
| the bold, that is.
|
| Think of a lot of the inflammatory content on social media,
| how people have made whole careers and fortunes over outrage,
| and they have no shame over it.
|
| It really does begin to look like having a good sense of
| shame isn't rewarded in the same way.
| postepowanieadm wrote:
| If you are from poor society you can't afford to have shame.
| You either succeed or fail, again and again, and keep trying.
| MrBuddyCasino wrote:
| In other news, wet roads cause rain.
| nobodywillobsrv wrote:
| I would imagine there are a lot of "small nice to haves" that
| people submit because they are frustrated about the mere
| complexity of submitting changes. Minor things that involve a
| lot of complexity merely in terms of changing some config or
| some default etc. Something where there is a significant
| probability of it being wrong but also a high probability of
| someone who knows the project being able to quickly see if it's
| ok or not.
|
| i.e. imagine a change that is literally a small diff, that is
| easy to describe as a mere user and not a developer, and that
| requires quite a lot of deep understanding merely to submit as
| a PR (build the project! run the tests! write the template for
| the PR!).
|
| Really a lot of this stuff ends up being a kind of failure mode
| of various projects that we all fall into at some point where
| "config" is in the code and what could be a simple change and
| test required a lot of friction.
|
| Obviously not all submissions are going to be like this but I
| think I've tried a few little ones like that where I would
| normally just leave whatever annoyance I have alone but think
| "hey maybe it's 10 min faff with AI and a PR".
|
| The structure of the project incentives kind of creates this.
| Increasing cost to contribution is a valid strategy of course,
| but from a holistic project point of view it is not always a
| good one especially assuming you are not dealing with
| adversarial contributors but only slightly incompetent ones.
| vanderZwan wrote:
| Kind of by definition we will not see the people who do not
| submit frivolous PRs that waste the time of other people. So
| keep in mind that there's likely a huge amount of survivor bias
| involved.
|
| Just like with email spam I would expect that a big part of the
| issue is that it only takes a minority of shameless people to
| create a ton of contribution spam. Unlike email spam these
| people actually want their contributions to be tied to their
| personal reputation. Which in theory means that it should be
| easier to identify and isolate them.
| bgro wrote:
| All email is spam.
| pil0u wrote:
| The major companies that made available the very tools they use
| to create this spam code, applied the exact same ethics.
| guerrilla wrote:
| > Other people apparently don't have this feeling at all.
|
| I think this is interesting too. I've noticed the difference in
| dating/hook-up contexts. The people you're talking about also
| end up getting laid more but that group also has a very large
| intersection with sex pests and other shitty people. The thing
| they have in common though is that they just don't care what
| other people think about them. That leads some of them to be
| successful if they are otherwise good people... or to become
| borderline or actual crininals if not. I find it fascinating
| actually, like how does this difference come about and can it
| actually be changed or is it something we get early in life or
| from the genetic lottery.
| lm28469 wrote:
| Lots of people cosplay as developers, and "contributing" to
| open source is a box they must check. It's like they go through
| the moves without understanding they're doing the opposite of
| what they should be doing. Same with having a tech blog, they
| don't understand that the end goal is not "having a blog" but
| "producing and sharing quality content"
| GardenLetter27 wrote:
| The Internet (and developer communities) used to be a high
| trust society - mostly academics and developers, everyone with
| shared experiences of learning when it was harder to get
| resources, etc.
|
| The grift culture has changed that completely, now students
| face a lot of pressure to spam out PRs just to show they have
| contributed something.
| quanwinn wrote:
| I'm not surprised. Lower barrier of entry -- thanks to AI in
| this case -- often leads to a decrease in quality in most
| things.
| slfreference wrote:
| https://x.com/JDHamkins/status/2014085911110131987
|
| I am seeing the doomed future of AI math: just received
| another set theory paper by a set theory amateur with an AI
| workflow and an interest in the continuum hypothesis.
|
| At first glance, the paper looks polished and advanced. It is
| beautifully typeset and contains many correct definitions and
| theorems, many of which I recognize from my own published
| work and in work by people I know to be expert. Between those
| correct bits, however, are sprinkled whole passages of claims
| and results with new technical jargon. One can't really tell
| at first, but upon looking into it, it seems to be
| meaningless nonsense. The author has evidently hoodwinked
| himself.
|
| We are all going to be suffering under this kind of garbage,
| which is not easily recognizable for the slop it is without
| effort. It is our regrettable fate.
| micromacrofoot wrote:
| It's because the AI is generating code better than they would
| write, and if you don't like it then that's fine... they didn't
| write it
|
| it's easy to not have shame when you have no skin in the
| game... this is similar to how narcissists think so highly of
| themselves, it's never their fault
| mg794613 wrote:
| It's because a lot of people that werent skilful werent on your
| path before. Now that pandora's box has been re-opened, those
| people feel "they get a second chance at life". It's not that
| they have no shame, they have no perspective to put that shame.
|
| You on the other hand, have for many years honed your craft.
| The more you learn, the more you discover to learn aka , you
| realize how little you know. They don't have this. _At all_.
| They see this as a "free ticket to the front row" and when we
| politely push back (we should be way harsher in this, its the
| only language they understand) all they hear is "he doesn't
| like _me_." which is an escape.
|
| You know how much work you ask of me, when you open a PR on my
| project, they don't. They will just see it as "why don't you
| let me join, since I have AI I should have the same skill as
| you".... unironically.
|
| In other words, these "other people" that we talk about haven't
| worked a day in the field in their life, so they simply don't
| understand much of it, however they feel they understand
| everything of it.
| nlh wrote:
| This is so completely spot on. It's happening in other fields
| too, particularly non-coding (but still otherwise specialized
| or technical) areas. AI is extremely empowering but what's
| happening is that people are now showing up in all corners of
| the world armed with their phone at the end of their
| outstretched arm saying "Well ChatGPT says..." and getting
| very upset when told that, no, many apologies, but ChatGPT is
| wrong here too.
| __loam wrote:
| It's why artists despise the AI art users. In that field it
| isn't simply them trying to contribute but instead
| insisting that you wasted your time learning to create art
| and if you're a professional you deserve to starve. All
| while being completely ignorant to the medium or the
| process.
| ThrowawayR2 wrote:
| It's not as if there weren't that sort of people in our
| profession even before the rise of LLMs, as evidenced by the
| not infrequent comments about "gatekeeping" and "nobody needs
| to know academic stuff in a real day-to-day job" on HN.
| alfalfasprout wrote:
| This is easily the most spot-on comment I've read on HN in a
| long time.
|
| The humility of understanding what you don't know and the
| limitations of that is out the window for many people now. I
| see time and time again the idea that "expertise is dead".
| Yet it's crystal clear it's not. But those people cannot
| understand why.
|
| It all boils down to a simple reality: you can't understand
| why something is fundamentally bad if you don't understand it
| at all.
| johnnyanmac wrote:
| That all makes sense. But the more I know, the more I realize
| that a lot of software engineering isn't about crazy
| algorithms and black magic. I'd argue a good 80% of it is the
| ability to pick up the broken glass, something even many
| students can pull off. 15% of that comes down to avoiding
| landmines in a large field as you pick up said glass.
|
| But that care isn't even evident here. People submitting prs
| that don't even compile, bug reports for issues that may not
| even exist. The minimum I'd expect is to check the work of
| whatever you vibe coded. We can't even get that. It's some.
| Odd form of clout chasing as if repos are a factor of
| success, not what you contribute to them.
| BlackjackCF wrote:
| I find that interesting because for the first 10 years of my
| career, I didn't feel any confidence in contributing to open
| source at all because I didn't feel I had the expertise to do
| so. I was even reluctant to file bugs because I always
| figured I was on the wrong and I didn't want to cause churn
| for the maintainers.
| Aurornis wrote:
| A subset of open source contributors are only interested in
| getting something accepted so they can put it on their resume.
|
| Any smart interviewer knows that you have to look at actual
| code of the contributions to confirm it was actually accepted
| and that it was a non-trivial change (e.g. not updating
| punctuation in the README or something).
|
| In my experience this is where the PR-spammers fall apart in
| interviews. When they proudly tell you they're a contributor to
| a dozen popular projects and you ask for direct links to their
| contributions, they start coming up with excuses for why they
| can't find them or their story changes.
|
| There are of course lazy interviewers who will see the resume
| line about having contributed to popular projects and take it
| as strong signal without second guessing. That's what these
| people are counting on.
| jm4 wrote:
| Some people just want their name in the contributor list,
| whether it's for ego, to build a portfolio, etc. I think that's
| what it comes down to. Many projects, especially high profile
| ones, have to deal with low effort contributions - correcting
| spelling mistakes, reformatting code, etc. It's been going on
| for a long time. The Linux contributor guidelines - probably a
| lot of other projects too - specifically call this stuff out
| and caution people not to do it lest they suffer the wrath of
| the LKML. AI coding tools open up all kinds of new
| possibilities for these types of contributors, but it's not AI
| that's the problem.
| OGEnthusiast wrote:
| > The biggest surprise to me with all this low-quality
| contribution spam is how little shame people apparently have.
|
| My guess is it's mostly people from countries with a culture
| that reward shameless behavior.
| kkukshtel wrote:
| This is one thing I find funny about all the discussion around
| AI watermarking. Yes for absolutely nefarious bad actors it is
| incredibly important, but what seems clear is that the majority
| of AI users do absolutely nothing to conceal obvious tells of
| AI generation. Turns out people are shameless!
|
| Two immediate ones I can think of:
|
| - The yellow hue/sepia tone of any image coming out of ChatGPT
|
| - People responding to text by starting with "Good Question!"
| or inserting hard-to-memorize-or-type unicode symbols like -
| into text where they obviously wouldn't have used that and have
| no history of using it.
| wnevets wrote:
| > how little shame people apparently have
|
| You can expand this sentiment to everyday life. The things some
| people are willing to say and do in public is a never ending
| supply of surprising.
| hintymad wrote:
| > The biggest surprise to me with all this low-quality
| contribution spam is how little shame people apparently have
|
| My guess is that those people have different incentives. They
| need to build a portfolio of open-source contributions, so
| shame is not of their concern. So, yeah, where you stand
| depends on where you sit.
| SpecialistK wrote:
| Funny enough, reading this makes me feel a little more
| confident and less... shame.
|
| I've been deep-diving into AI code generation for more niche
| platforms, to see if it can either fill the coding gap in my
| skillset, or help me learn more code. And without writing my
| whole blog post(s) here, it's been fairly mediocre but
| improving over time.
|
| But for the life of me I would never submit PRs of this code.
| Not if I can't explain every line and why it's there. And in
| preparation of publishing anything to my own repos I have a
| readme which explicitly states how the code was generated and
| requesting not to bother any upstream or community members with
| issues from it. It's just (uncommon) courtesy, no?
| cranium wrote:
| A well crafted policy that, I think, will be adopted by many OSS.
|
| You'd need that kind of sharp rules to compete against unhinged
| (or drunken) AI drivers and that's unfortunate. But at the same
| time, letting people DoS maintainers' time at essential no cost
| is not an option either.
| Lucasoato wrote:
| > Bad AI drivers will be banned and ridiculed in public. You've
| been warned. We love to help junior developers learn and grow,
| but if you're interested in that then don't use AI, and we'll
| help you. I'm sorry that bad AI drivers have ruined this for you.
|
| Finally an AI policy I can agree with :) jokes aside, it might
| sound a bit too agressive but it's also true that some people
| have really no shame into overloading you with AI generated shit.
| You need to protect your attention as much as you can, it's
| becoming the new currency.
| weinzierl wrote:
| I don't think ridicule is an effective threat for people with
| no shame to begin with.
| wpietri wrote:
| Well, this is explicitly public ridicule. The penalty isn't
| just feeling shamed. It's reputational harm, immortalized via
| Google.
|
| One of the theorized reasons for junk AI submissions is
| reputation boosting. So maybe this will help.
|
| And I think it will help with people who just bought into the
| AI hype and are proceeding without much thought. Cluelessness
| can look a lot like shamelessness at first.
| mijoharas wrote:
| I think it makes sense, both for this, and for curl.
|
| Presumably people want this for some kind of prestige, so
| they can put it on their CV (contributed to ghostty/submitted
| security issue to curl).
|
| If we change that equation to have them think "wait, if I do
| this, then when employers Google me they'll see a blog post
| saying I'm incompetent" changes calculation that is
| neutral/positive for if their slop gets accepted to
| negative/positive.
|
| Seems like it's addressing the incentives to me.
| Applejinx wrote:
| Doubly so as these bad AI drivers are trading away even the
| possibility of having attention. It's very possible to render
| yourself senseless through a habit of deference. Even if you're
| coming up with ways to optimize AI responses, you are just
| trying to make a more superior superior to defer to.
| rikschennink wrote:
| > No AI-generated media is allowed (art, images, videos, audio,
| etc.). Text and code are the only acceptable AI-generated
| content, per the other rules in this policy.
|
| I find this distinction between media and text/code so
| interesting. To me it sounds like they think "text and code" are
| free from the controversy surrounding AI-generated media.
|
| But judging from how AI companies grabbed all the art, images,
| videos, and audio they could get their hands on to train their
| LLMs it's naive to think that they didn't do the same with text
| and code.
| embedding-shape wrote:
| > To me it sounds like "text and code" are free from the
| controversy surrounding AI-generated media.
|
| It really isn't, don't you recall the "protests" against
| Microsoft starting to use repositories hosted at GitHub for
| training their own coding models? Lots of articles and
| sentiments everywhere at the time.
|
| Seems to have died down though, probably because most
| developers seemingly at this point use LLMs in _some_ capacity
| today. Some just use it as a search engine replacement, others
| to compose snippets they copy-paste and others wholesale don 't
| type code anymore, just instructions then review it.
|
| I'm guessing Ghostty feels like if they'd ban generated
| text/code, they'd block almost all potential contributors. Not
| sure I agree with that personally, but I'm guessing that's
| their perspective.
| rikschennink wrote:
| Right, that's what I'm thinking too (I'll update my statement
| a bit to make that more clear), but I constantly hear this
| perspective that it's all good for text and code but when
| it's media, then it's suddenly problematic. It's equally
| problematic for text and code.
| Applejinx wrote:
| I bet they aren't honoring the terms of the MIT license I use
| for my repos. It's pretty lenient and I bet they're still not
| compliant.
| embedding-shape wrote:
| And to be frank, why would they? Who would stop them? Would
| take a massive case for them to be compelled to be stopped,
| and no one seems to care about attribution anymore, or
| licensing at all in most cases. Companies are using
| torrents to download copyrighted material, stuff
| individuals gone to prison for before, and they hardly even
| get a slap on the wrist.
| Applejinx wrote:
| I see a downvote. All righty then: cite where they credited
| me. Go on :)
| NiloCK wrote:
| It's not that code is distinct or "less than" art. It's an
| authority and boundaries question.
|
| I've written a fair amount of open source code. On anything
| like a per-capita basis, I'm way above median in terms of what
| I've contributed (without consent) to the training of these
| tools. I'm also specifically "in the crosshairs" in terms of
| work loss from automation of software development.
|
| I don't find it hard to convince myself that I have moral
| authority to think about the usage of gen AI for writing code.
|
| The same is not true for digital art.
|
| There, the contribution-without-consent, aka theft, (I could
| frame it differently when I was the victim, but here I can't)
| is entirely from people _other than me_. The current and future
| damages won 't be born by me.
| rikschennink wrote:
| Alright, if I understand correctly, what you're saying is
| they make this distinction because they operate in the "text
| and code" space but not in the media space.
|
| I've written _a lot_ of open source MIT licensed code, and
| I'm on the fence about that being part of the training data.
| I've published it as much for other people to use for
| learning purposes as I did for fun.
|
| I also build and sell closed source commercial JavaScript
| packages, and more than likely those have ended up in the
| training data as well. Obviously without consent. So this is
| why I feel strong about making this separation between code
| and media, from my perspective it all has the same problem.
| Applejinx wrote:
| re: MIT license, I generally tell people they have to
| credit and that's functionally the only requirement. Are
| they crediting? That's really the lowest imaginable bar,
| they're not asked to do ANYTHING else.
| antirez wrote:
| TLDR don't be an asshole and produce good stuff. But I have the
| feeling that this is not the right direction for the future.
| Distrust the process: only trust the results.
|
| Moreover this policy is strictly unenforceable because good AI
| use is indistinguishable from good manual coding. And sometimes
| even the reverse. I don't believe in coding policies where
| maintainers need to spot if AI is used or not. I believe in
| experienced maintainers that are able to tell if a change looks
| sensible or not.
| b3kart wrote:
| This doesn't work in the age of AI where producing crappy
| results is much cheaper than verifying them. While this is the
| case, metadata will be important to understand if you should
| even bother verifying the results.
| antirez wrote:
| The time needed for an AI patch written with the prompt "now
| make it as small as possible, clean, and human coded" is as
| big as reviewing the patch itself.
| sumtechguy wrote:
| As someone who has picked up recently some 'legacy' code. AI
| has been really good at mostly summing up what is going on. In
| many cases it finds things I had no idea was wrong (because I
| do not know the code very well yet). This is so called 'battle
| hardened code'. I review it and say 'yeah its is wildly broken
| and I see how the original developer ended up here'. Sometimes
| the previous dev would be nice enough to leave a comment or
| some devs 'the code is the comments'. I have also had AI go
| wildly off the rails and do very dumb things. It is an
| interesting tool for sure one you have to keep an eye on or it
| will confidently make a foot gun for you. It is also nice for
| someone like me who has some sort of weird social anxiaty thing
| about bugging my fellow devs. In that I can create options
| tables and pick good ideas out of that.
| danw1979 wrote:
| I'm not sure I agree it's completely unenforceable: a sloppy,
| overly verbose PR, maybe without an attached issue, is pretty
| easy to pick out.
|
| There's some sensible, easily-judged-by-a-human rules in here.
| I like the spirit of it and it's well written (I assume by
| Mitchell, not Claude, given the brevity).
| senko wrote:
| On a tangent: the origin of the problems with low-quality drive-
| by requests is github's social nature. That might have been great
| when GitHub started, but nowadays many use it as portfolio
| padding and/or social proof.
|
| "This person contributed to a lot of projects" heuristic for
| "they're a good and passionate developer" means people will
| increasingly game this using low-quality submissions. This has
| been happening for years already.
|
| Of course, AI just added kerosene to the fire, but re-read the
| policy and omit AI and it still makes sense!
|
| A long term fix for this is to remove the incentive.
| Paradoxically, AI might help here because this can so trivially
| be gamed that it's obvious it's not longer any kind of signal.
| stephantul wrote:
| Your point about rereading without ai makes so much sense.
|
| The economics of it have changed, human nature hasn't. Before
| 2023 (?) people also submitted garbage PRs just to be able to
| add "contributed to X" to their CV. It's just become a lot
| cheaper.
| TeMPOraL wrote:
| Let's not forget the Hacktober Fest, the scourge of open
| source for over a decade now, the driver of low-quality
| "contribution" spam by hordes of people doing it _for a
| goddamn free t-shirt_.
|
| No, this problem isn't fundamentally about AI, it's about
| "social" structure of Github and incentives it creates (fame,
| employment).
| achyudh wrote:
| Mailing lists essentially solve this by introducing friction:
| only those who genuinely care about the project will bother to
| git send-email and defend a patch over an email thread. The
| incentive for low-quality drive-by submissions also evaporates
| as there is no profile page with green squares to farm. The
| downside is that it potentially reduces the number of
| contributors by making it a lot harder for new contributors to
| onboard.
| evilhackerdude wrote:
| sounds reasonable to me. i've been wondering about encoding
| detailed AI disclosure in an SBOM.
|
| on a related note: i wish we could agree on rebranding the
| current LLM-driven never-gonna-AGI generation of "AI" to
| something else... now i'm thinking of when i read the in-game
| lore definition for VI (Virtual Intelligence) back when i played
| mass effect 1 ;)
| KolmogorovComp wrote:
| A good PR using IA should be impossible to distinguish from a
| non-AI one.
| layer8 wrote:
| The problem is that most aren't good, and bad ones can take a
| lot of effort to distinguish, if they look plausible on the
| surface. So the potentially good ones aren't worth all the bad
| ones.
| KolmogorovComp wrote:
| I agree with most of them being bad, I disagree with them
| taking lots of effort to distinguish, and I am maintainer
| unfortunately receiving receiving more and more using AI.
| epaga wrote:
| I really like the phrase "bad AI drivers"...AI is a tool, and the
| stupid drive-by pull requests just mean you're being
| inconsiderate and unhelpful in your usage of the tool, similar to
| how "bad drivers" are a nightmare to encounter on a highway...so
| stop it or you'll end up on the dashcam subreddit of programming.
| skybrian wrote:
| The experience of using a coding agent is that you're more of a
| "backseat driver" though. The AI acts as your driver and you
| tell it where to go, sometimes making corrections if it's going
| the wrong way.
| cmsj wrote:
| The experience is what you make of it. Personally I'm quite
| enjoying using AI as a way to generate code I can disagree
| with and refactor into what I want.
| skybrian wrote:
| Sure, and I ask it to do the refactoring, too!
| KronisLV wrote:
| > You must state the tool you used (e.g. Claude Code, Cursor,
| Amp)
|
| Interesting requirement! Feels a bit like asking someone what IDE
| they used.
|
| There shouldn't be that meaningful of a difference between the
| different tools/providers unless you'd consistently see a few
| underperform and would choose to ban those or something.
|
| The other rules feel like they might discourage AI use due to
| more boilerplate needed (though I assume the people using AI
| might make the AI fill out some of it), though I can understand
| why a project might want to have those sorts of disclosures and
| control. That said, the rules themselves feel quite reasonable!
| milancurcic wrote:
| I think that a warning of public ridicule may be fine. However,
| actually doing it is quite low brow IMO. I'm sad to see more and
| more otherwise admirable projects step down to that (assuming
| they actually do it).
| phanimahesh wrote:
| An unenforced threat is toothless. Publicly stating we do not
| appreciate XYZ pr that was ai generated, low effort and in bad
| faith is perfectly acceptable.
| milancurcic wrote:
| I agree with the second statement but public ridicule can be
| understood to have a stronger effect, and be in bad faith
| itself.
| njhnjhnjh wrote:
| I'm happy to see it. Fuck these AI imbeciles. They should live
| in fear of having their reputations destroyed.
| hmokiguess wrote:
| Banned I understand but ridiculed? I would say that these bad
| drive by spammers are analogous to phishing emails. Do you engage
| with those? Are they worth any energy or effort from you? I think
| ghostty should just ghost them :)
|
| EDIT: I'm getting downvoted with no feedback, which is fine I
| guess, so I am just going to share some more colour on my opinion
| in case I am being misunderstood
|
| What I meant with analogous to phishing is that the intent for
| the work is likely the one of personal reward and perhaps less of
| the desire to contribute. I was thinking they want their name on
| the contributors list, they want the credit, they want something
| and they don't want to put effort on it.
|
| Do they deserve to be ridiculed for doing that? Maybe. However, I
| like to think humans deserve kindness sometimes. It's normal to
| want something, and I agree that it is not okay to be selfish and
| lazy about it (ignoring contribution rules and whatnot), so at
| minimum I think respect applies.
|
| Some people are ignorant, naive, and are still maturing and
| growing. Bullying them may not help (thought it could) and
| mockery is a form of aggression.
|
| I think some true false positives will fall into that category
| and pay the price for those who are truly ill intended.
|
| Lastly, to ridicule is to care. To hate or attack requires caring
| about it. It requires effort, energy, and time from the
| maintainers. I think this just adds more waste and is more
| wasteful.
|
| Maybe those wordings are there just to 'scare' people away and
| maintainers won't bother engaging, though I find it is just
| compounding the amount of garbage at this point and nobody
| benefits from it.
|
| Anyways, would appreciate some feedback from those of you that
| seem to think otherwise.
|
| Thanks!
|
| PS: What I meant with ghostty should "ghost" them was this:
| https://en.wikipedia.org/wiki/Shadow_banning
| gverrilla wrote:
| shaming doesn't work.
| lifetimerubyist wrote:
| Very quaint of them to exempt themselves because they've "proven
| themselves" already.
|
| Surely _they_ are incapable of producing slop because they are
| just so much smarter than everyone else so the rules shouldn 't
| apply to them, surely.
| yomismoaqui wrote:
| Until now code was something costly to make and could only be
| created by our monkey brains.
|
| But now we have some kind of electronic brains that can also
| generate code, not at the level of the best human brains out
| there but good enough for most projects. And they are quicker and
| cheaper than humans, for sure.
|
| So maybe in the end this will reduce the need for human
| contributions to opensource projects.
|
| I just know that as a solo developer AI coding agents enable me
| to tackle projects I didn't think about event starting before.
| Sparkyte wrote:
| It is important to write the code yourself so you understand how
| it functions. I tried vibe coding a little bit. I totally felt
| like I was reading someone else's code base.
|
| Sanitization practices of AI are bad too.
|
| Let me be clear nothing wrong with AI in your workflow, just be
| an active participator in your code. Code is not meant to be one
| and done.
|
| You will go through iteration after iteration, security fix after
| fix. This is how development is.
| PlatoIsADisease wrote:
| These are unbelievably reasonable terms for AI.
|
| I might copy it for my company.
| dw_arthur wrote:
| I think a social norm of disclosing AI use at all times would be
| great. People and companies should also be held 100% accountable
| for anything created using AI.
| twoodfin wrote:
| I wish HN had a title tag for AI-generated posts, like it used
| to have for PDF's and still does for year-of-publication.
| tzs wrote:
| > No AI-generated media is allowed (art, images, videos, audio,
| etc.). Text and code are the only acceptable AI-generated
| content, per the other rules in this policy
|
| What's the reason for this?
|
| Media is the most likely thing I'd consider using AI for as part
| of a contribution to an open source project.
|
| My code would be hand crafted by me. Any AI use would be similar
| to Google use: a way to search for examples and explanations if
| I'm unclear on something. Said examples and explanations would
| then be read, and after I understand what is going on I'd write
| my code.
|
| Any documentation I contributed would also be hand written.
| However, if I wanted to include a diagram in that documentation I
| might give AI a try. It can't be worse than my zero talent
| attempts to make something in OmniGraffle or worse a photograph
| of my attempt to draw a nice diagram on paper.
|
| I'd have expected this to be the least concerning use of AI.
| andy99 wrote:
| > Issues and discussions can use AI assistance but must have a
| full human-in-the-loop. This means that any content generated
| with AI must have been reviewed and edited by a human before
| submission.
|
| I can see this being a problem. I read a thread here a few weeks
| ago where someone was called out on submitting an AI slop article
| they wrote with all the usual tells. They finally admitted it but
| said something to the effect they reviewed it and stood behind
| every line.
|
| The problem with AI writing is at least some people appear
| incapable of critically reviewing it. Writing something yourself
| eliminates this problem because it forces you to pick your words
| (there could be other problems of course).
|
| So the AI-blind will still submit slop under the policy but
| believe themselves to have reviewed it and "stand behind" it.
| hereme888 wrote:
| At the moment I have 20 subagents fixing stuff throughout my own
| codebase.
|
| But I've never had the gall to let my AI agent do stuff on other
| people's projects without my direct oversight.
| alya wrote:
| At the Zulip open-source project, we've had a significant
| onslaught of AI slop in the past few months. It gets as absurd as
| PR descriptions with AI-generated "screenshots" of the app to
| "demonstrate" the changes. We've had to start warning
| contributors that we won't be able to review their work if they
| continue misusing AI, and occasionally banning repeat offenders.
| It's feels draining -- we want to spend our time mentoring
| _people_ who 'll actually learn from feedback, not interacting
| with contributors who are just copy-pasting LLM responses without
| thought.
|
| Our evolving AI policy is in the same spirit as ghostty's, with
| more detail to address specific failure modes we've experienced:
| https://zulip.readthedocs.io/en/latest/contributing/contribu...
| verdverm wrote:
| Honestly, yours looks nothing like Mitchell's to me, and that's
| a good thing.
|
| It's actually reasonable and the guidance you provide on how to
| best use Ai when contributing to Zulip is :chef's kiss:
|
| truly, I'm going to copy yours as a thank you!
| krzyk wrote:
| It always amazes me why image/videos/audio generated by AI are
| treated differently from code.
|
| Are images somehow better? If one draws, is he better the one
| that writes code? Why protect one and not the other. Or why
| protect any form at all?
| zzzeek wrote:
| the endless issue with coding policy statements like this is that
| the people who need to read them the most are the ones who
| couldn't care less and don't read anything.
| yellowapple wrote:
| This is the most well-informed and reasonable AI policy I've seen
| so far. Neither kneejerk-hostile nor _laissez faire_ ; just a
| mature understanding of the limitations of LLMs, and an
| insistence on transparency and accountability when using them.
| waldrews wrote:
| We're just not going to see any code written entirely without AI
| except in specialist niches, just as we don't see handwritten
| assembly and binaries. So the disclosure part is going to become
| boilerplate.
|
| In the old era, the combination 'it works' + 'it uses a
| sophisticated language' + 'it integrates with a complex codebase'
| implied that this was an intentional effort by someone who knew
| what they were doing, and therefore probably safe to commit.
|
| We can no longer make that social assumption. So then, what can
| we rely on to signal 'this was thoroughly supervised and reviewed
| and understood and tested?' That's going to be hard and
| subjective.
|
| Personal reputations and track records are pedigrees and brands
| are going to become more important in the industry; and the
| meritocratic 'code talks no matter where you came from' ethos is
| at risk.
| BoredomIsFun wrote:
| How about autocomplete with LLMs? Should it be disclosed too?
| (scratching my balding head).
___________________________________________________________________
(page generated 2026-01-23 23:01 UTC)