[HN Gopher] I'm tired of fixing customers' AI generated code
___________________________________________________________________
I'm tired of fixing customers' AI generated code
Author : BitWiseVibe
Score : 428 points
Date : 2024-08-21 23:16 UTC (17 hours ago)
(HTM) web link (medium.com)
(TXT) w3m dump (medium.com)
| w_for_wumbo wrote:
| I'd be tempted to just roll with the AI generated
| endpoints/hallucinations. If it's presenting me statistically
| probable names that make sense after absorbing the world's
| knowledge, I'm tempted to lean into that, instead of insisting
| that I have it right. Correct names don't succeed as often as
| useful names do.
| pocketarc wrote:
| That's a great idea, but I think the main problem is that the
| generated endpoints/properties will be affected quite heavily
| by whatever the prompt/context was.
|
| AI isn't necessarily saying "this is the one endpoint that will
| always be generated". Unless it is - if the customer generated
| code is always the same endpoints/properties then it'd
| definitely make sense to also support those.
| TillE wrote:
| I'm always a little surprised at how many people out there want
| to develop software yet haven't put in the effort to gain even
| the most basic computer nerd programming chops. You see this all
| the time in the more newbie-friendly game engine communities.
|
| Maybe you don't want to pursue a career in software, but anyone
| can spend a week learning Python or JavaScript. I suspect/hope a
| lot of these people are just kids who haven't gotten there yet.
| jprete wrote:
| Coding requires a willingness to understand and manipulate
| systems made of unbreakable rules. Most people don't want to
| deal with such an uncompromising method of communication.
| busterarm wrote:
| And then there are those of us who find computers to be more
| bearable than people...
|
| At least computers don't think it's their god-given right to
| treat you like garbage.
| Mistletoe wrote:
| Did you not use Windows 95?
| __MatrixMan__ wrote:
| What percentage of web traffic today would you say is
| composed of bits that the user--if they bothered to inspect
| it--would prefer to not have anything to do with? I'd say
| it's more than half.
|
| Computers treat people like garbage all the time.
| fragmede wrote:
| you're right, but I have a hard time picturing the
| computer as having emotions to be able to treat me like
| garbage in the first place. you won't rm the file? sudo
| rm file! the computer could fight back and say access
| denied still because of extended attributes, but for some
| reason I don't equate poorly written software as being
| treated like garbage. I always imagine some hapless
| programmer is doing the best they could with the
| resources they have in the system they're under, it's
| just not very good but that's not their fault.
| __MatrixMan__ wrote:
| I'm similarly sympathetic when I come across a buggy
| implementation. It's malicious design that I'm objecting
| to. But I suppose it's a bit silly to say that it's the
| computer that's treating me like garbage. It's just that
| someone else is in control of my computer, and they're
| treating me like garbage.
| sim7c00 wrote:
| sadly these days, it seems a keen mind is only a
| machine's mind. People spend far to little time to
| understand what they are telling their poor computers to
| do. And look what happens, people start turning against
| them. Blaming them for their misdoing. It's like Blaming
| god, the government, a nation, family or a tribe. These
| are all made of humans. Human bad behavior is at the core
| of our suffering. Nothing else.
| __MatrixMan__ wrote:
| I don't disagree substantively, but I do think there are
| uniquely modern aspects to the question of "am I enabling
| bad behavior right now?". It's not just ethics, it's
| education.
|
| Consider for instance the remote support features that
| Intel is so keen on advertising these days. Microcode
| level remote access is a small help for IT departments
| and a huge help for authoritarian regimes looking to spy
| on their people. But I don't think that most people are
| prepared to consider what they're enabling by paying
| Intel to continue to grow into a telescreen vendor.
|
| Sure, we shouldn't blame the computer's soul for bad
| behavior. But if it's being used as a weapon, it's not
| helpful to remove the computer from the conversation and
| say "well it's actually bad people." Mitigating bad
| behavior via computer means hacking that computer, and
| that starts with blaming it for the bad behavior to some
| degree.
| sim7c00 wrote:
| its not computers who treat people like garbage. they do
| as they are instructed by humans, in all cases.
| foobarchu wrote:
| Counterintuitively, it also requires a willingness to break
| what appear at first to be unbreakable rules. Most of the
| worst programmers I know seem to see their work as "how can I
| accomplish a task without breaking what I see as the rules",
| without having fully understood the system. That quickly
| turns into copypasta, extra layers of abstraction, over
| configurability, and many of the other plagues of
| programming.
| jprete wrote:
| I think the two errors - imagining non-existent rules, and
| ignoring rules that exist - are related errors. The
| foundational skill is accepting that the machine is never
| wrong because the machine is also never "right", the
| machine doesn't actually make decisions, it's a construct
| of physics following a pile of physical laws and not a
| person to be negotiated with.
| hamandcheese wrote:
| I think you overestimate the amount of skill a newb can quickly
| gain on their own. I taught myself to code, but it took a whole
| summer (aka free time that adults don't get) and I had access
| to my dad (who was a software engineer himself) to answer lots
| of questions.
| dylan604 wrote:
| I read a "Teach yourself $language in 10 days" book in a
| weekend, and was banging code on Monday to create the first
| v0.1 in a week. Of course the code was absolutely horrendous,
| but it worked. I still have a copy of that old database that
| was used, and over the years, I have turned to it as I've
| learned new things and have even rewritten the UI a couple of
| times. It has helped me stay up to date with new trends as it
| was originally written in '99 using frames, then went to full
| CSS/JS, then used it to learn flex, and so on.
|
| So, if you're solo dev'ing, you can get away with making
| things work with what you've learned in a week. You just
| wouldn't be hired by anyone else of a serious nature. So it
| just depends on the individual and projects being worked.
| malfist wrote:
| But you're building on the context of knowing a different
| language.
|
| Picking up the second, fifth, or nth language is easy, as
| long as it isn't the first one.
| kloop wrote:
| The second one actually seems to be harder for some
| people. It requires separating the syntax from the
| semantics
|
| Agreed on all further ones, however
| dylan604 wrote:
| What makes you say that? I learned HTML using Notepad and
| Netscape. I had a single semester as senior in high
| school that taught PASCAL, but that was 7 years prior.
| Not really sure how that helped in the slightest.
|
| I don't feel this is any different from someone that
| might have taken a class that taught HTML/JS/CSS except
| for that would actually be learning directly applicable
| to today. If that type of person jumped into a bootcamp,
| I feel like that would be similar to anything I
| experienced if not better. The internet is a thing now so
| there is so much more access to anything I had.
| mulmen wrote:
| I strongly disagree with this. It's more like languages
| hit a common wavelength. Sometimes it makes sense,
| sometimes it doesn't.
| FredPret wrote:
| If I may make some assumptions about you:
|
| 1. You're a person of a particular frame of mind who finds
| it easy and natural to talk to computers in programming
| languages
|
| 2. You knew a different language before, perhaps one you
| learnt at a young age
|
| 3. You've messed around with computers for years now and
| have built up a conceptual model of what the hardware and
| software components are and how things fit together. So if
| a new thing comes along, you can hang it on your tree of
| knowledge. Consider the difficulty someone might have
| making hello_world.py if they don't know what an OS is, or
| how to edit text, or any of the basics.
|
| None of the above generalizes to the population at large.
| fragmede wrote:
| not "at large", but there's very much a segment of smart
| people who's expertise lies elsewhere, and they just
| haven't taken the time to learn the basics of
| programming. as someone who's spent a lot of time
| programming, I love meeting brilliant people who could
| program but don't just because they've gone a different
| way.
| dylan604 wrote:
| Some people forget how easy programming can be when you
| know nothing and just try stuff to see what works.
| Working in a procedural manner with everything in a
| global scope is simple to get stuff working. Not
| everything has to be extrapolated out into namespaces,
| functions, classes. It's nightmare code to maintain
| later, but going from blank page to working code is
| totally possible.
|
| I think sometimes we forget not everything has to be
| written to a git repo with a highly developed structure
| ready for multiple people to work on. Is it a good habit,
| hellznaw, but people start somewhere and progress. That
| was the point that I was trying to make. It is totally
| possible to have a career as a programmer and have no
| credentialed degrees in CS or even programming. I know
| from personal experience.
| throwaway2037 wrote:
| I agree with you. In the 2000s, what you describe was
| normal for Excel/VBA. The trick to learn VBA from
| nothing, was to use the macro recorder, then slowly
| modify the code. And, arguably, Excel formulas was/is
| functional programming.
| skydhash wrote:
| I learned C by using gcc directly and Python by using
| IDLE. Both with single file project. The actual software
| engineering can take time, but simple projects are very
| easy for beginner to reason about. Everyone can build a
| shed, it's building a house that requires professional
| expertise.
| raincole wrote:
| Perhaps you're a genius then.
|
| I am quite sure most people who have only learned
| programming for one weekend would write much worse code
| than ChatGPT.
| dylan604 wrote:
| What part of "the code was horrendous" did not click with
| your sentiment? It was horrible. The entire database was
| one table. Every SQL query was a SELECT *, and filtered
| everything downstream in the code rather than WHERE. It
| was absolutely horrible code that I am shocked actually
| worked with any kind of speed that actually felt
| responsive. Of course I didn't have millions of records,
| but the fact that it worked at all was encouraging enough
| to me that I'm still doing it to this day in the same
| language. Only now I've been doing it for 20+ years and
| I'm much less embarrassed about my code, or maybe more.
| At least back then I could use "I'm a beginner" as an
| excuse.
| busterarm wrote:
| I've been programming since I was 4 years old but with
| literally zero resources or assistance beyond "take books out
| from the library" for 20 years. It wasn't until I was about
| 28 until I had the chops to get into the industry (largely
| down to never having a need or opportunity to learn SQL --
| also I mean as a developer, I had a prior career in IT) and
| even then it wasn't until I was 31 before I had the
| confidence enough to interview...
|
| On the other hand, I have a wealth of other general computer
| and protocol knowledge and have been working circles around
| most of my coworkers since day one. In the typical tech
| startup world I _rarely_ encounter coworkers with truly deep
| knowledge outside of whatever language they work in.
|
| IMO the skill isn't about being able to "write code", it's
| about being able to model how things work.
| datavirtue wrote:
| This is very similar to my story. Once I got on a dev team
| I was flabbergasted at the lack of breadth and depth of
| knowledge of my fellow devs. It was only a few older devs
| that had any clue.
|
| Having gobs of time as a kid and in my twenties to
| experiment greatly enhanced my capabilities. Once I did
| start landing corporate jobs (which was exceedingly
| difficult) I was at or above architect level. As I gained
| more experience working on production systems I was
| promoted very quickly (created new positions for me etc).
|
| I have had other architects declare I was the best they
| have ever met. Which sadly, isn't saying much.
| busterarm wrote:
| > Once I did start landing corporate jobs (which was
| exceedingly difficult) I was at or above architect level.
| As I gained more experience working on production systems
| I was promoted very quickly (created new positions for me
| etc).
|
| Ditto and ditto.
|
| I have had some positive experiences working with fresh
| grads from places like Waterloo (I'd hire 10 of their
| grads for any one grad from anywhere else...) but my
| professional experience very much matches yours.
| sph wrote:
| Sorry, are you saying it takes 20 years for a self-taught
| developer to learn enough programming, especially when they
| start as kids? No offense, this is a you problem.
|
| I dabbled with PCs since I was 8, around 14 I had enough
| brain to start to understand BASIC and enough free time to
| get good enough to write half a decent mini OS by the time
| I was 17 [1] and got my first paying job (sysadmin and PHP
| dev) at 19. I'm 37 now.
|
| All you need is free time and being interested enough in
| the subject matter. And kids learn 10x as fast as adults
| anyway.
|
| Not sure why you are trying to discourage people from
| learning on their own based on your time line.
|
| 1: https://github.com/1player/klesh
| jazzyjackson wrote:
| wow its crazy that different people have different
| experiences
|
| (as it turns out, writing code for yourself is a
| different skillset than writing code for a boss!)
| busterarm wrote:
| > (as it turns out, writing code for yourself is a
| different skillset than writing code for a boss!)
|
| That was exactly the point I was trying to make
| tristor wrote:
| Similar experience, except I have never worked
| professionally writing software as my primary task. I've
| always stayed in operations/systems roles or other
| periphery roles, and now am a PM. I am constantly amazed at
| how many "senior" engineers actually have no understanding
| about how a computer actually works. Once I went corporate
| I moved up the ranks on the systems side of things very
| fast, and was widely regarded as one of the best engineers
| in the company everywhere I went, and yet I can see
| absolute chasms in my knowledge and really try to ensure I
| identify SMEs I can work with to overcome my own gaps. It
| is really shocking though how little most working engineers
| actually understand about technology.
| steve1977 wrote:
| The expectation of ,,quickly gain" is the problem.
| mrbombastic wrote:
| A week is not anywhere close to enough to learn programming in
| any meaningful way for someone with no experience.
| tourmalinetaco wrote:
| It is, however, enough to make small programs and extend from
| there. Especially following a book like K&R.
| wuming2 wrote:
| Time to expertise is down to zero in "Fake until you make it"
| circles. 260 week-long iterations later, having survived the
| "hype curve" and the "Valley of Death", they declare
| themselves "battle-proven". A.k.a. experts.
| sergiotapia wrote:
| the worst is when someone knows all the keywords to make it
| seem like they are technical but after talking for a few
| days you realize wait they really don't know wtf they're
| talking about!
| whatshisface wrote:
| This is one of the rewards for paying full attention to
| people, even when you aren't forced to by the situation:
| small misalignments slip out long before you hear
| something that jars you into a critical frame of mind.
| hinkley wrote:
| Expert beginners.
| jpc0 wrote:
| Until I decided to start "reinventing the wheel" and just not
| using abstractions from popular libraries and frameworks I
| really struggled to actually understand what is happening.
|
| I feel like a week isn't anywhere near close enough but
| depending on what you want to do it gets you to start
| tinkering. Ironically I do wish that I had started working on
| embedded with microcontrollers than starting with web purely
| because there isn't space for absurd abstractions.
|
| On web even the DOM API is a huge abstraction over rendering
| calls to OpenGL/DirectX/Vulkan and I never could grok what is
| happening with the DOM API until I played with the underlying
| tech and learnt about trees and parsers and how that would be
| stored.
|
| I still use the DOM and love the abstraction, but sometimes I
| just wish I could use an immediate mode approach instead of
| the retained mode that the DOM is...
|
| Someone with a week of knowledge, or even someone who has
| spent 10 years building react may not understand half of that
| unless they have actively tried to learn it. Thwy might have
| an idea if they had formal education but a self taught
| programmer. They have been building houses using lego blocks,
| I you give them mortar and bricks you are setting them up for
| failure.
| Workaccount2 wrote:
| Ironically I learned programming by playing with
| microcontrollers, which I got into through learning about
| electronics. So I had a really true "ground up" learning
| experience, starting with embedded C (not machine code, I
| wasn't that hard). I did a number of projects on AVR's and
| got decent at writing programs.
|
| When moved on to writing PC programs, I struggled so much
| because everything is so heavily abstracted and languages
| like python have so much ability embedded in them already.
| I kinda had to toss a lot of intuition and learn things
| new.
| gorbachev wrote:
| The other thing is that to work on anything really meaningful
| takes time and effort. It takes determination to struggle
| through that in the beginning when you're running into one
| problem after another.
| nsonha wrote:
| > You see this all the time in the more newbie-friendly game
| engine
|
| games tend to attract young people (read: beginners) but at the
| same time game programming's barrier to entry is pretty high
| with maths & physics, low-level graphical programming, memory
| management, low level language and dependencies, OOP... It's
| almost obvious that this should be the case, every kid who's
| interested to coding I talked to wants to do something with
| games.
| elzbardico wrote:
| This is not the case any more and have not been for a very
| long time. There are plenty of game engines, and some of them
| are specifically targeting beginning game devs and abstract a
| lot of that stuff in really high level concepts that require
| no much more from the developer than some really basic
| arithmetics and geometry intuition.
|
| In fact, there are so many beginner-friendly gaming engines
| out there for most languages, that I am convinced that we
| should start using games as the entry-point for teaching
| programming languages. It is a beatifully self-contained
| domain.
| soared wrote:
| Game dev is a fun hobby some people like to mess around with,
| just like any other hobby. Doesn't mean they'll be experts or
| know what they're doing, but they'll try and probably ask some
| basic questions online.
|
| I mess around in goody and game maker and can write some shitty
| code there, but I've never written a line of code for work. I
| just like messin around for fun
| sim7c00 wrote:
| its nice to have a hobby. I make Operating systems as a
| hobby. But because I am not an Army of dilligent engineers
| with a knowledge based in all history of computing and
| computer science, i would not dare to ship my code to an
| unwitting user and allow them to connect to the internet.
| Thats dangerous in these times.
|
| Do you know how many botnets exist because of shit game-
| engines that are easily exploited and connected peer-to-peer
| etc.
|
| Lovely frameworks and engines... but really harmful if you
| ask me. People are unwitting victims of other peoples hobby
| projects.
|
| You need to be responsible in this day and age. If you don't
| want to do the due dilligence, or are stapped for resources
| (time, knowledge, etc.) then it's best to for example, make a
| singleplayer game, or LAN only and disallow any IP addresses
| in your game not defined as internal class ranges.
|
| Do have a hobby, and do have fun, but do so responsibly. You
| wouldn't want one of your works of love and passion to end up
| hurting someone would you? Simple steps can be taken to
| project the consumers of your lovecraft from the vicious
| world out there. It's sad this is needed, but that's no
| excuse not to do it. Humans should be better, but they are
| not.
| dylan604 wrote:
| There's a common phrase that founders should code, but not all
| founders are coders. So when the start up is small and the
| founders want to contribute by testing PoCs, the chatbots are
| getting used by those founders that can't code. Lucky for me,
| the PoC is just that and allowed to be implemented without
| shimming the PoC directly.
|
| I cringe every time they mention using the bots, but luckily it
| has been controllable.
| triyambakam wrote:
| I think people often don't know where to begin.
| fingerlocks wrote:
| Agreed. I've experienced this in different programming
| domains.
|
| A mere ten or so years ago I only wrote firmware in C and
| MacOS apps in objective-c. That was my world and it was all I
| knew, and all I've done for a long time.
|
| Then something happened. Small startup. Website needs urgent
| fix and the web guy is MIA, so what the hell, I can take a
| stab at it.
|
| Literally had no idea where to start. I didn't know about
| npm, minified transpiling, much less actual testing and
| deployment. Could not make sense of anything anywhere.
| Hopelessly lost. I even grepped the JavaScript for "void
| main()" out of desperation. Just ridiculous
| viccis wrote:
| One of my favorite intern stories was a kid who was a compsci
| senior, very good university, and who was assigned to write
| some Python code for my team. He had the very immature "Python
| is a baby's language" attitude. He wasn't working on my stuff
| so I don't really keep track of what he's doing, but a few
| weeks later I look at what he has written. Almost all of his
| Python functions have one parameter with an asterisk, and he
| does a len() check on it, printing and return an integer if
| it's not the right length of function arguments. Turns out this
| guy learned this behavior from Perl, used an asterisk because
| why not he always does in C, and was just manually unpacking
| every function argument and using a C style return error
| handling process.
|
| Still the most insane thing I've seen, but I know there are a
| lot of kids out of college who got used to gen AI for code
| writing who put out a lot of this kind of code. Also,
| coincidentally, we haven't hired any US college interns in
| about 3 years or so.
| fragmede wrote:
| A friend of mine says LLMs are good at producing bad pandas
| code because there's just so much of it out there to train
| off of.
| giraffe_lady wrote:
| This sort of thing is a weird relic of CS programs doing
| double duty as "professional school for software development"
| and "undergrad prep for an academic math career."
|
| You just don't know how much they actually learned about
| programming as a discipline in its own right and it very well
| could be functionally zero. I've seen recent CS grads who
| didn't know how to use git, didn't know how to split code
| across multiple files or understand _why you would even want
| to_.
|
| I think there's a fairly sound argument for these being
| different degrees, that a certain kind of researcher doesn't
| necessarily need these skills. But since it isn't there's
| just a huge range in how schools reconcile it.
| creesch wrote:
| I am not, I see it happening even within companies. They figure
| that for some junior tech related roles they don't need to hire
| people with the education and just teach them in house. Often
| not developing itself, but things like automated tests in a QA
| role.
|
| The result is people that have no technical background, no real
| interest in it either, no basic framework to start from
| learning to use a specific set of tools and a very basic
| understanding of programming.
| stavros wrote:
| > I'm always a little surprised at how many people out there
| want to develop software yet haven't put in the effort to gain
| even the most basic computer nerd programming chops.
|
| If you're surprised by reality, that says something about your
| mental model, not about reality. People don't want to "learn
| programming", they want to "make a thing". Some people learn
| programming while making a thing, but why learn about how the
| sausage is made when all you want is to eat it?
| sfn42 wrote:
| I like learning how to do stuff. It's strange to me that
| people think they can do stuff without learning how
| dahart wrote:
| > People don't want to "learn programming", they want to
| "make a thing".
|
| Hahaha, I wish that were true, but it's not. Lots of people
| want to learn programming for learning and programming's
| sake, or because programming is more lucrative than sausage
| making. I think I've worked with more programmers that care
| more about programming than the end result, than making
| something. It's constantly frustrating and a huge source of
| over-engineering mistakes that have proliferated through
| engineering.
|
| > why learn about how the sausage is made when all you want
| is to eat it?
|
| Then why do sausages get made? It's because not everyone only
| wants to eat them. There's a variety of reasons people make
| sausages and _also_ like eating them, from making money, to
| making high quality or interesting variety sausages, to being
| self-sufficient and not paying others for it, to learning
| about how it's done. It's been my experience that the more
| someone cares about eating sausages, the more likely they are
| to dabble in making their own.
| port19 wrote:
| Anyone who already programs for a couple of years can spend a
| week learning $lang. Learning programming for the first time
| takes a long while and a lot of effort. I'd say a couple of
| months if you're bright and motivated. Possibly a year or two
| if you're not.
| rurp wrote:
| I agree with the part that someone who wants to build something
| technical should gain at least some related knowledge, but a
| week is underselling the effort needed to learn how to code by
| a lot. After one week of teaching myself Python I couldn't code
| my way out of a paper bag, and I'm someone who enjoyed it
| enough to stick with it. The average person would need at least
| 10x that amount of time to be able to start building something
| interesting.
| tensor wrote:
| I'm not surprised at all. Honestly the "I don't need to learn
| that" mentality is common in tech even in people who call
| themselves senior developers. It's especially noticeable in the
| hostility of many towards the sorts of information you learn in
| a good computer science degree.
|
| How many arguments have we heard here along the lines of "why
| teach algorithms universities should be teaching
| _insert_fad_technology_of_the_day_." Big Oh and time complexity
| is a special favourite for people to pick on and accuse of
| being useless or the like. You see it in arguments around SQL
| vs document databases, people not being willing to recognize
| that their for loops are in fact the same as performing joins,
| people unwilling to recognize that yes they have a data schema
| even if they don't write it down.
|
| So I'm not surprised at all that people would use AI as a
| substitute for learning. Those same people have likely gotten
| by with stackoverflow copypasta before gen AI came about.
| randomdata wrote:
| Seems like the natural progression from end goal to breaking it
| down into the smaller and smaller pieces required to see the
| goal through, as people have always done.
|
| Before LLMs you'd probably have to reach for learning Python or
| Javascript sooner, at least if StackOverflow didn't have the
| right code for you to copy/paste, but I expect anyone who
| sticks with it will get there eventually either way.
| tazu wrote:
| Hilariously, the target market for the author's API seems to be
| the same as the top post on HN today[0]: "traders".
|
| I think amateur "trading" attracts a specific brand of idiot that
| is high/left on the Dunning Kruger curve. While taking money from
| idiots is a viable (even admirable) business strategy, you may
| want to fully automate customer service to retain your sanity.
|
| [0]: https://news.ycombinator.com/item?id=41308599
| cscurmudgeon wrote:
| Out of topic, but:
|
| https://www.mcgill.ca/oss/article/critical-thinking/dunning-...
|
| > The two papers, by Dr. Ed Nuhfer and colleagues, argued that
| the Dunning-Kruger effect could be replicated by using random
| data.
| kaoD wrote:
| The more I'm reading this article the less I understand their
| point. Is there an actual paper that describes how their
| "random data" is generated?
|
| I can also generate random data that looks like any
| distribution by carefully choosing the random distribution.
| What's their point?
| bofadeez wrote:
| It's people who generally don't have any other skills and
| reject all evidence for Efficient Market Hypothesis. They
| legitimately think what they're doing is not gambling. No
| amount of empirical evidence can convince them they have no
| risk-adjusted alpha
| creesch wrote:
| Yeah the customer demographic here likely does worsen the
| situation. Although I am sure that this is happening elsewhere
| as well.
| cratermoon wrote:
| There's already a big market for taking out AI garbage, and it
| expect it to grow as the AI bubble bursts. The best thing a
| consultant can do today is learn the common issues and failure
| modes of AI generated code.
|
| Providing an API service to customers means you will get terrible
| client code. Sometimes it's dumb stuff like not respecting rate
| limits and responding to errors by just trying again. One option,
| if you don't want to fix it yourself, is to partner with a
| consultant who will take it on, send your customers to them. Bill
| (or set your API access rates) appropriately.
|
| Sometimes you have to fire customers. Really bad ones that cost
| more than they bring in are prime candidates for firing, or for
| introducing to your "Enterprise Support Tier".
| djaouen wrote:
| I'm not! Email in bio.
| matrix_overload wrote:
| Well, if you are annoyed by a particular maintenance task related
| to your business, find a way to automate it!
|
| In this case, you could create examples for your API in common
| programming languages, publish them on the product website, and
| even implement an automatic test that would verify that your last
| commit didn't break them. So, most of the non-programmer
| inquiries can be answered with a simple link to the examples page
| for the language they want.
|
| As a bonus point, you will get some organic search traffic when
| people search for <what your API is doing> in <language name>.
| mrbombastic wrote:
| Funnily enough an llm is pretty good at categorizing and
| prioritizing support requests.
| lagniappe wrote:
| There's room for all of us in this industry. What someone is
| unwilling to do is just an opportunity for someone else to pick
| up the yoke.
| aunty_helen wrote:
| This could actually be an ingenious way of solving the problem.
| If someone has a support issue and can't solve it themselves,
| yet requires coding help, forward them a freelancer that they
| can hire for 20$/hr from upwork that knows this API well etc.
| fragmede wrote:
| Unfortunately it doesn't look like there's a way to contact
| the author of this piece.
| nkrisc wrote:
| I have the sense that most of these people won't be willing to
| pay anything to have their code fixed.
| shadowgovt wrote:
| I wonder if there's any consistent pattern to the API
| hallucinations.
| surfingdino wrote:
| Yes. They are all crap.
| lqcfcjx wrote:
| now you probably should build an AI to fix AI generated code
| based on your documentation.
| bdcravens wrote:
| This seems like a support issue, not an AI issue. AI is how the
| code was written, but the issue would be the same if it was
| amateurs writing bad code. If all you want to do is support your
| API, a support article outlining the issues you see over and over
| would be something to point your customers to. Warrant your API
| against errors, but point out that anything more is billable
| work. If you're not interested, partner with someone to do that
| work. You could even offer support contracts that include some
| amount of customization.
| SpicyLemonZest wrote:
| It's a support issue in a sense, but in many contexts people
| _want_ to offer a better support experience than "anything
| more is billable work". A reputation for being helpful and
| customer-friendly is valuable, especially in a business where
| you're selling to other programmers, and you can't buy that
| reputation after the fact with money from support contracts.
| omoikane wrote:
| This post seems to be saying that AI opened up a new avenue for
| people to demand free work.
|
| If someone asked "I wrote some rough designs and interfaces,
| can you write me an app for free?" The author could easily
| detect it as a request for free work. But because the first
| part is hidden behind some ChatGPT generated code and the
| second part is disguised as a request for help, the author
| would be tricked into doing this free work, until they detected
| this pattern and write a blog post about it.
| xoac wrote:
| Is this a chatgpt summary of the article?
| m463 wrote:
| > amateurs writing bad code
|
| in volume, this turns into support writing the code.
|
| I think of how the south park movie folks sent so much
| questionable content to the censors that the compromise in the
| end let through lots of the puppet sex.
| prisenco wrote:
| That's like saying "seems like the problem is the internet is
| filled with low quality content" in response to ai bots when,
| while not wrong, the new problem is that we've created a way to
| accelerate the creation of that low quality content many orders
| of magnitude faster.
|
| So what was a difficult problem can quickly become
| insurmountable.
| fzeroracer wrote:
| The difference is scale. I don't know how many times people
| need to say this, but LLM tools enable people to spam low
| quality code at a rate that is far faster than ever.
|
| There's been multiple stories and posts here on HN about issues
| with AI generated PRs for open source repos because of people
| using them to game numbers or clout. This is a similar problem
| where you have a bunch of people using your API and then
| effectively trying to use you for free engineering work.
| paretoer wrote:
| Totally agree the difference is in the scaling properties.
|
| On the other hand, I look around the room I am in and it is
| filled mostly with "low quality" Chinese made products. While
| you can't compare these products to expensive, expertly
| crafted, high end products, there is another scaling law at
| play when it comes to price. The low quality Chinese products
| are so cheap that I don't even consider the more expensive
| options. When the low quality desk lamp is close enough and
| 20x cheaper than the well made desk lamp, there is no
| decision point for me.
|
| If it breaks, I will just buy another one.
| nsonha wrote:
| is there a way we can make a support bot or a chat-based document
| that is fine tuned and limit the answer to only what's in the
| API? Getting the users to use it is another issue but one problem
| at a time.
| protocolture wrote:
| Yeah so every tech revolution does this right.
|
| ATMs were meant to kill banking jobs but ah theres more jobs in
| banking than ever.
|
| The Cloud was meant to automate away tech people, but all it did
| was create new tech jobs. A lot of which is cleaning up after
| idiots who think they can outsource everything to the cloud and
| get burned.
|
| LLMs are no different. The "Ideas Man" can now get from 0 to 30%
| without approaching a person with experience. Cleaning up after
| him is going to be lucrative. There are already stories about
| businesses rehiring graphic designers they fired, because someone
| needs to finish and fine tune the results of the LLM.
| dylan604 wrote:
| I seriously hope those rehires are coming back with a refined
| rate as well.
| protocolture wrote:
| A healthy dickhead tax on the way back in would be smart.
| busterarm wrote:
| > ATMs were meant to kill banking jobs but ah theres more jobs
| in banking than ever.
|
| ATMs only handle basic teller functions and since COVID in NYC
| I had to change banks twice because I couldn't find actual
| tellers or banks with reasonable open hours. BoA had these
| virtual-teller only branches and the video systems were always
| broken (and the only option on Saturday). This was in Midtown
| Manhattan and my only option was basically a single branch on
| the other side of town 8-4 M-F.
|
| I'm now happily with a credit union but at least since moving
| to the south things are generally better because customers
| won't tolerate not being able to deal with an actual person.
| creesch wrote:
| > ATMs were meant to kill banking jobs but ah theres more jobs
| in banking than ever.
|
| US banks, who are surprisingly behind the times in as far as
| automation goes. Here a lot of banks used a lot of automation
| to reduce the amount of manual jobs needed. to the degree that
| many offices are now also closing as everything can be done
| online.
|
| And no, there is no need to visit banks here as I get the
| impression it is in the US. We don't even have physical checks
| anymore.
| reportgunner wrote:
| Sadly everyone thinks they are The "Ideas Man".
| anonzzzies wrote:
| I have a business fixing broken code/systems (especially if it is
| stressful and last minute); if you are tired/annoyed of something
| in the software market, just up your fees. For us not much
| changed; a lot of badly (throw the spec over the wall) outsourced
| software was fairly bad since forever; AI generated code is
| similar. I guess this will grow even faster though, as normally
| solid developers might take on a lot more work and get sloppy
| with AI.
| sira04 wrote:
| Could you make a model from your API/docs and let them plug that
| into their AI stuff? That would be funny.
| danielmarkbruce wrote:
| If you are building an API and have decent docs, it's a totally
| ok trade off to say "i'll lose some customers this way, but I'm
| not providing support". And just be upfront about it. Some stores
| have a no return policy with no exceptions. They lose some
| customers, it's ok.
| tonyoconnell wrote:
| Why don't you use AI to provide support? I'm serious actually.
| This sounds like something AI can do really well.
| hsbauauvhabzb wrote:
| Spoken like someone whose never been on the receiving end of
| any such 'support'.
| ilaksh wrote:
| For the problems given in the article, it will 100% work. It's
| very easy for Claude 3.5 or gpt-4o to look at documentation for
| a couple of API endpoints, compare it to submitted code, and
| point out invalid endpoints and properties. It can provide
| correct code also if the custom is asking for something
| possible.
|
| It won't be flawless but if the issues are as basic as stated
| in this article, then it seems like a good option.
|
| It will cost money to use the good models though. So I think
| the first step would be to make sure if they ask for an invalid
| endpoint it says that in so many words in the API response, and
| if they ask for an invalid property it states that also.
|
| Then if that doesn't give them a clue, an LLM scans incoming
| support requests and automatically answers them. Or flags them
| for review.. To prevent abuse, it might be a person doing it
| but just pressing a button to send an auto-answer.
|
| But maybe a relatively cheap bot in a Discord channel or
| something connected to llama 3.1 70b would be good enough to
| start, and people have to provide a ticket number from that bot
| to get an answer from a person.
| stitched2gethr wrote:
| I'm a bit torn. My first thought was "If the current state of
| the art LLMs made the mistakes it's unlikely an LLM would be
| able to correct them." But I'm not sure that's true if the
| support LLM (chat bot) is given very specific instructions so
| as to limit the possible answers. Still I think that's gonna
| break down pretty quick for other reasons.
|
| Maybe the chat bot can recognize a botched request and even
| suggest a fix but what then? It certainly won't be able to
| convert the user's next request into a working application of
| even moderate complexity. And do you really want a chat bot to
| be the first interaction your customers have.
|
| I think this is why we haven't seen these things take off
| outside of very large organizations who are looking to save
| money in exchange for making customers ask for a human when
| they need one.
| FeepingCreature wrote:
| > I'm a bit torn. My first thought was "If the current state
| of the art LLMs made the mistakes it's unlikely an LLM would
| be able to correct them."
|
| But, I mean, that doesn't make sense even for humans, right?
| 99% of the errors I make, I can easily correct myself because
| they're trivial. But you still have to go through the process
| of fixing them, it doesn't happen on its own. Like, for
| instance, just now I typoed "stil" and had to backspace a few
| letters to fix it. But LLMs cannot backspace (!), so you have
| to put them into a context where they feel justified in going
| back and re-typing their previous code.
|
| That's why it's a bit silly to make LLM code, try to run it,
| see an error and immediately run to the nearest human. At
| least let the AI do a few rounds of "Can you spot any
| problems or think of improvements?" first!
| fragmede wrote:
| The first time I asked ChatGPT to write a function, have it
| run it and repeat until the function meets given test
| cases, in one shot, was pretty cool.
| mediumsmart wrote:
| I always make gpt fix the code it created. How else would I
| learn.
| alexeiz wrote:
| I had a related episode at work when my coworker asked me why his
| seemingly trivial 10 line piece of code was misbehaving
| inexplicably. It turned out he had two variables `file_name` and
| `filename` and used one in place of another. I asked him how he
| ended up with such code, he said he used copilot to create it.
| Using code from a generative AI without understanding what it
| does is never a good idea.
| frumper wrote:
| I knew a guy that made a good living as a freelance web
| developer decades ago. He would pretty much just copy and paste
| code from tutorials or stack overflow and had no real idea how
| anything worked. Using code without understanding it is never a
| good idea, it doesn't need to be from AI for that to be true.
| Laakeri wrote:
| But he made a good living out of it, so in the end it was a
| good idea?
| prisenco wrote:
| It certainly puts a ceiling on a career. And I'd argue it
| probably gave him a pretty rough shelf life. At some point
| he has to understand what he's doing.
|
| Unless he's so good at selling his services he can
| consistently find new clients. And if that's the case, he'd
| probably kill it in sales.
| fragmede wrote:
| sales engineer is quite a lucrative career. don't have to
| be really good at it, just enough to be useful.
| snoxy wrote:
| Sales engineers have to be good enough to bluff their way
| through the layers of hyperbole/minor exaggeration/utter
| bullshit (delete as applicable) the sales team have spun.
| Whether their conscience gets involved before the deal
| closes, different question.
| efilife wrote:
| Cope. People often make money on things they know nothing
| about
| dazzawazza wrote:
| I'll bet the ceiling is CTO.
| thephyber wrote:
| He may have made a good living, but his customer / employer
| bought low quality code with lots of tech debt.
|
| That business model only works until customers are
| sophisticated enough to understand tech debt. In the
| future, more customers will be less willing to pay the same
| good wages for low quality code.
| throwaway2037 wrote:
| > but his customer / employer bought low quality code
| with lots of tech debt.
|
| Sarcastic reply: Isn't that most tech? Even good (above
| average) developer produce lots of tech debt and
| sometimes low quality code.
| datavirtue wrote:
| Yeah, and the business people could not care less. I am
| on a team taking in millions of dollars from a Delphi
| Windows app from 1997. Zero tests, horribly mangled
| business logic embedded in UI handlers. Maintaining that
| app is not feasible. I'm rebuilding a modern version of
| it only because it is embarrassing to demo and is such a
| UX nightmare that our distributor made us commit to a new
| app.
| bbarnett wrote:
| "Webdev" makes me think of wordpress, which is like
| planting 20 onions in your backyard, and comparing
| yourself to a farmer with acres of crops.
|
| I can completely believe someone had no idea what they
| were doing when copy/pasting, and working on wordpress.
| f6v wrote:
| Or maybe you're just exaggerating. I've done my fair share of
| copy pasting and it never worked to just do it without
| understanding what's going on.
|
| I think the problem with "AI" code is that many people have
| almost a religions belief. There're weirdos on internet who
| say that AGI is couple years away. And by extension current
| AI models are seen as something incapable of making a mistake
| when writing code.
| n4r9 wrote:
| The other downside to AI code vs stackoverflow is that a
| stackoverflow post can be updated, or a helpful reply might
| point out the error. With the advent of LLMs we may be
| losing this communal element of learning and knowledge-
| sharing.
| Piskvorrr wrote:
| We aren't. LLMs may have been useful for a moment in
| time, _before_ the trick "it's now MY OWN creation, no
| IP strings attached - when it comes through the
| plagiarism machine" became apparent, and _before_ the
| models started eating their own tail. Now they 're just
| spiralling down, and it will IMNSHO take something else
| than an iterative "a future version will surely fix this,
| One Day, have faith."
| n4r9 wrote:
| There are signs of a decline in people asking and
| answering questions on sites like stack exchange:
| https://meta.stackexchange.com/questions/387278/has-
| stack-ex...
|
| So I hope you're right, but the evidence is currently
| that you're wrong. Let's see how it plays out, I suppose.
| Piskvorrr wrote:
| - Which _might_ be a different matter: of specifically SE
| declining. (A very different, and long-running, tragedy,
| but one that began long before the current AI boom and
| prompted by very different, non-technical issues.)
|
| - That said, surely traffic _will_ decline for Q &A
| sites. "How do I connect tab A into slot B" is something
| that people are likely to query LLMs for; the response
| will surely sound authoritative, and could be even
| correct. That's definitely a task where LLMs could help:
| common questions that have been asked many times (and as
| such, are likely to be well-answered in the human-made
| training data). A 20001st question of "how do I right-
| align a paragraph in HTML" has not been posted? Good.
| Rote tasks are well-suited to automation. (Which, again,
| brings us back to the issue "how to distinguish the
| response quality?")
| jtbayly wrote:
| But what happens with the next generation of questions?
| The reason LLMs can answer how to right-align a paragraph
| in HTML is at least in part because it has been asked and
| answered publicly so many times.
|
| Now imagine that HTMZ comes along and people just go
| straight to asking how to full justify text in HTMZ for
| their smart bucket. What happens? I doubt we'll get good
| answers.
|
| It feels like the test of whether LLMs can stay useful is
| actually whether we can stop them from hallucinating API
| endpoints. If we could feed the rules of a language or
| API into the LLM and have it actually reason from that to
| code, then my posed problem would be solved. But I don't
| think that's how they fundamentally work.
| Cthulhu_ wrote:
| This is a known issue from like the 2000s where there were so
| many bad PHP tutorials, a lot of SQL injection and XSS etc
| came from those.
| jazz9k wrote:
| I knew someone similar. They would just get free templates
| and sell them as a website to customers, with almost no
| changes, aside from logos and text. Most had no Javascript or
| css and looked terrible, even by 2005 standards.
|
| His clients were usually older small business owners that
| just wanted a web presence. His rate was $5000/site.
|
| Within a few years, business dried up and he had to do
| something completely different.
|
| He also hosted his own smtp server for clients.It was an old
| server on his cable modem in a dusty garage. I helped him
| prevent spoofing/relaying a few times, but he kept tinkering
| with the settings and it would happen all over again.
| falcor84 wrote:
| >a guy that made a good living ... never a good idea
|
| Arguably the term for a bad idea that works is "good idea"
| frumper wrote:
| That's fair. From a personal perspective it was a good
| idea. He regularly had sites get compromised though, so for
| his customers it wasn't always a good product. He generally
| kept his customers happy though.
| digging wrote:
| Or maybe "good" and "bad" aren't useful descriptors in this
| context.
| macksd wrote:
| There are plumbers who make a living but whose work results
| in leaks in people's homes. They're making a living, but I
| don't consider the way they work "a good idea".
| JKCalhoun wrote:
| At least AI comments their code.
| mooreds wrote:
| > Using code from a generative AI without understanding what it
| does is never a good idea.
|
| Hear hear!
|
| I feel like genAI is turning devs from authors to editors.
| Anyone who thinks the latter is lesser than the former has not
| performed both functions. Editing properly, to elevate the
| meaning of the author, is a worthy and difficult endeavor.
| FanaHOVA wrote:
| Copilot wouldn't make a typo. He just made that up and / or
| broke the code himself.
| creesch wrote:
| LOL, are you kidding me? LLMs including the gpt family
| copilot uses are very good at sneaking in these sorts of
| thing.
| FanaHOVA wrote:
| Show me 1 example.
| creesch wrote:
| Show me that copilot never makes a mistake or introduces
| variables that never have been initialized...
| brigadier132 wrote:
| This is a strawman, he never said it didn't make
| mistakes.
| creesch wrote:
| Oh for crying out loud, I obviously mean these specific
| mistakes. If you have worked in any capacity with LLMs
| like this you would have seen them variables or suddenly
| switch up the convention of how they're written.
|
| Certainly if you are in a conversation mode after a few
| back and forths this happens from time to time.
|
| I am just not going to spend my time digging to previous
| prompts of code I might not want to share just to satisfy
| a random internet person .
| brigadier132 wrote:
| The models I've used don't make typos on variable names
| that already exist in the context. Typos are not the
| failure mode, this is literally the easiest text
| prediction task they can do.
| sixfiveotwo wrote:
| What you guys probably want to do instead is get to a
| common definition of what a typo is. Personally, I
| understand it as a typographic error, which is a fancy
| way of saying a spelling mistake (a mistake on a letter),
| not a mistake where one use a word for another.
|
| Maybe you meant the latter?
| Mashimo wrote:
| I, for one, have not have this experience with LLM
| creating new variable names when they already defined
| one.
|
| Lots of mistakes, but never this one.
| shakna wrote:
| There's more than one mistake in the code given, and I
| tried just once.
|
| "Create Python code for iterating a file with a cache."
| class CachedFileIterator: def __init__(self,
| file_path): self.file_path = file_path
| self.cache = [] # List to store cashed lines
| self.current_index = 0 # Index for the current line to
| read def _read_lines_from_file(self):
| """Read lines from the file and cache them."""
| with open(self.file_path, 'r') as openFile:
| for line in f:
| self.cache.append(line.strip()) # Strip newline
| characters and save to cache def
| __iter__(self): """Return the iterator
| object itself.""" return self
| def __next__(self): """Return the next
| line in the iteration.""" if
| self.current_index < len(self.cache):
| result = self.cache[self.current_index]
| self.current_index += 1 return result
| else: # If we reach the end of the
| cache, we read more lines from the file if necessary
| if len(self.cache) == 0: # Cache is empty, read file
| self._read_lines_from_file()
| if self.current_index < len(self.cache):
| result = self.cache[self.current_index]
| self.current_index += 1 return
| result else:
| # If still no lines, we stop iteration
| raise StopIteration # Example usage:
| if __name__ == "__main__": file_path =
| 'example.txt' # Replace with your file path
| iterator = CacheingFileIterator(file_path)
| for line in iterator: print(line)
| yifanl wrote:
| Garbage code is bad enough, but it's not like people have
| never had to walk juniors through mistakes before LLMs.
|
| But this is actually so much worse for that same reason -
| the type of developer who'd submit Copilot output (I can
| call it that, as it's definitely not code) for a PR is
| unable to respond to any comment beyond asking Copilot
| again and wasting everyone's time with 6 more rounds of
| reviews. I've literally had to write test cases for
| someone else and told them "You can't ask for another
| code review until your code passes these."
| cutemonster wrote:
| > "You can't ask for another code review until your code
| passes these."
|
| Such a good idea :-) Maybe for job applications too and
| any at home work sample tests
| creesch wrote:
| Bit of a tangent, though related. It looks like you
| accidentally stumbled into a version of test driven
| development ;)
|
| With the big difference obviously being that typically
| the developer who writes the test also will write the
| code.
|
| In some situations, this actually makes sense to do with
| junior developers as part of their training. Where a
| senior developer sits down with them and write out the
| tests together, then with the tests as a guide they are
| thrown into the waters to develop the functionality.
|
| Of course, I suspect that in this case, you were not
| dealing with a junior. Rather the sort of person who
| looks at your tests, still is confused and asks for a
| "quick call" to talk about the tests.
| viraptor wrote:
| What do you see as mistakes? I see some weirdness, but
| the spec is just not complete - there was no requirement
| for rewinding, multiple users, etc. in the request so
| it's not implemented.
|
| The only thing I'd call an actual mistake is using an
| empty list to mean both an empty file and an
| uninitialised value.
| yifanl wrote:
| for line in f:
|
| is multiple mistakes in a single line.
| falcor84 wrote:
| What are the mistakes there?
| galbar wrote:
| The most obvious one: with
| open(self.file_path, 'r') as openFile: for
| line in f:
|
| `f` does not exist. It should be `openFile`.
| dpassens wrote:
| One is that the variable is called openFile and not f. I
| don't know enough python to see something else wrong with
| that but would love to know too, since I've written such
| a line just last week.
| yifanl wrote:
| f doesn't refer to anything.
|
| iterating over the file object at all instead of just
| calling self.cache = openFile.readlines() means that
| calling strip() the line below removes data beyond just
| the trailing newlines.
| shakna wrote:
| The file object is named "openFile", but used as "f". The
| class is defined as "CachedFileIterator", but used as
| "CacheingFileIterator". That's two typos, before
| discussing the actual code.
| fragmede wrote:
| would someone invent that and bother the author with that? I
| mean I suppose it's possible, but that seems like such a
| waste of time to me that I find that more unlikely. and while
| it's a typo, it's not fleinaem or something that's totally
| wrong, just a choice in breaking up the word filename. having
| written file handling code, the various permutations of
| filename and path and dirname get to be a bit much sometimes.
| brigadier132 wrote:
| People are unhinged about AI so yes I think someone would
| invent a scenario like this for internet points
| wredue wrote:
| >people are unhinged about AI.
|
| Well. We definitely agree on that.
| nope1000 wrote:
| I don't know about copilot but I've seen typos from ChatGPT
| (although it was english, not code)
| brigadier132 wrote:
| You are getting downvoted but you are right, a typo in a
| variable that already exists in a file like this is not the
| failure mode for LLMs. The failure mode is logic bugs, making
| up methods / functions.
| varjag wrote:
| No, you can get variable names "mutated" on follow up
| requests. The thing is like sculpting with toothpaste.
| brigadier132 wrote:
| I've been using copilot for as long as it has existed and
| what you are describing has not happened to me once.
| Literally on in the background 8 hours a day. Excuse me
| for not trusting the internet hivemind that hates
| everything that is hyped just a little bit.
| varjag wrote:
| My goto check of AI assistants is asking to write a
| function calculating the first N digits of Pi in Common
| Lisp. On at least two attempts when prompted to fix its
| code the model would change one of the variable names to
| T, which is a reserved symbol. So yeah pretty sure it
| does happen.
| boredhedgehog wrote:
| What likely happened is that he asked the AI two separate
| questions and fused the answers himself.
| Glyptodon wrote:
| At least for me stupid bugs like this turn out to be some of
| the most time wasting to debug, no AI involved. Like
| accidentally have something quoted somewhere, or add an 's' to
| a variable by accident and I may not even correctly process
| what the error message is reporting at first. Always feel a bit
| silly after.
| Noumenon72 wrote:
| These kinds of problems are often avoidable by linters or
| asking ChatGPT what is wrong, though I was just tearing my
| hair wondering why TSC_COMPILE_ERROR wasn't skipping
| TypeScript because I spelled it TSX_COMPILE_ERROR in my
| environment variable.
| viraptor wrote:
| Not only asking ChatGPT what is wrong, but also using an
| agent which does self-reflection by default. I'm sad every
| time I see people using the bare chat interface to generate
| code. We've got API tools which are so much better at it
| today. Use Aider at the very least.
| floydnoel wrote:
| does aider have an executable installer yet? i tried
| installing it but the python experience is terrible. last
| time i messed with python installs on my mac everything
| worked like shit until o reinstalled the OS.
| viraptor wrote:
| python -mvenv aider aider/bin/pip install aider-
| chat aider/bin/aider
|
| And you're done. There's also a docker version
| https://aider.chat/docs/install/docker.html
|
| Just don't mess with the system-wide installed version of
| python and it will be fine. This isn't a python specific
| issue though.
| arcticfox wrote:
| This type of bug is trivial for GPT to fix though. It was
| born for this. Sometimes it does generate real footguns but
| this sounds like an example from an earlier generation of
| generative AI.
| EVa5I7bHFq9mnYK wrote:
| Sounds like javascript "code". A normal language with proper
| type system would not allow that.
| CalRobert wrote:
| I don't see how typing relates to this.
| mkl wrote:
| Probably one of the variables is undefined, and static
| typing could catch that.
| timeon wrote:
| Also warning for unused variable.
| normie3000 wrote:
| Is it the compiler that would catch it?
| mkl wrote:
| Yes. This would be caught by the compiler in C, C++,
| Java, Rust, Haskell, etc.:
| https://stackoverflow.com/questions/1517582/what-is-the-
| diff...
|
| Many statically typed languages do have escape hatches to
| do some dynamic typing at runtime, but this is not the
| default (hence the classification), and it requires some
| additional effort to use.
| zo1 wrote:
| It's not the language, it's the IDE and laziness. They're
| doing this in notepad or maybe VSCode and don't have
| anything configured for highlighting or checking. Heck
| they probably don't even know how to interpret the error
| message saying "file_name is not declared".
|
| I'm the first to bash JS, but this is not a JS issue.
| It's 100% a "bad and lazy" human actor that is throwing
| spaghetti on the wall to see what sticks. In this case,
| they have a minigun cannon called ChatGPT/CoPilot that is
| letting them do more of it than what they used to.
| delusional wrote:
| We hired a new guy at work. In one of his first tasks he had
| chosen to write some bash, and it was pure nonsense. I mean it
| contained things like:
|
| if [ -z "${Var}+x" ]
|
| Where I can see what the author was trying to do, but the code
| is just wrong.
|
| I dont mind people not knowing stuff, especially when it's
| essentially Bash trivia. But what broke my heart was when I
| pointed out the problem, linked to the documentation, but
| recieved the response "I dont know what it means, I just used
| copilot" followed by him just removing the code.
|
| What a waste of a learning opportunity.
| woctordho wrote:
| Everyone working with shell scripts should know shellcheck
| swah wrote:
| And Python...
| rwmj wrote:
| And you didn't sack him?
| delusional wrote:
| I don't have hiring privileges. Either way. I like the guy,
| and I'd rather work to build him up. That doesn't mean it's
| not frustrating, but I have a process that seems to build a
| pretty good culture.
| OJFord wrote:
| And of a salary...
| steelframe wrote:
| Wait until a manager who's evaluating a technical decision
| you're making copies and pastes ChatGPT's "analysis" of your
| proposal and asks you to respond to it.
| falcor84 wrote:
| I agree that it's a waste of a learning opportunity, but from
| my experience it is still often rational.
|
| There were many times in my career when I had what I expected
| to be a one-off issue that I needed a quick solution for and
| I would look for a quick and simple fix with a tool I'm
| unfamiliar with. I'd say that 70% of the time the thing "just
| works" well enough after testing, 10% of the time it doesn't
| quite work but I feel it's a promising approach and I'm
| motivated to learn more in order to get it to work, and in
| the remaining 20% of the time I discover that it's just
| significantly more complex than I thought it would be, and
| prefer to abandon the approach in favor of something else; I
| never regretted the latter.
|
| I obviously lose a lot of learning opportunities this way,
| but I'm also sure I saved myself from going down many very
| deep rabbit holes. For example, I accepted that I'm not going
| to try and master sed&awk - if I see it doesn't work with a
| simple invocation, I drop into Python.
| Tainnor wrote:
| And any decent IDE will highlight a variable that is declared
| but unused. We already have "artificial intelligence" in the
| form of IDEs, linters, compilers, etc. but some people
| apparently think we should just throw it all away now that we
| have LLMs.
| tomrod wrote:
| No need for quotes, the best AI integrations are the ones you
| see as just part of the tech stack like spell check and
| linters.
| sa-code wrote:
| Unless you're using Python and said variable was meant to be
| reassigned, but you used a different name instead. E.g.
| file_name = 1 filename = 2
| aidos wrote:
| Fairly sure the linters would catch that (unless you
| referenced both of them in later code).
| ben_w wrote:
| > Using code from a generative AI without understanding what it
| does is never a good idea.
|
| True, but the anecdote doesn't prove the point.
|
| It's easy to miss that kind of difference even if you wrote the
| code yourself.
| planb wrote:
| In my experience this is exactly the kind of mistake an AI
| would _not_ make.
| yawnxyz wrote:
| Claude gave me something similar, except these were both used,
| and somehow global variables, and it got confused about when to
| use which one.
|
| Asking it to refactor / fix it made it worse bc it'd get
| confused, and merge them into a single variable -- the problem
| was they had slightly different uses, which broke _everything_
|
| I had to step through the code line by line to fix it.
|
| Using Claude's still faster for me, as it'd probably take a
| week for me to write the code in the first place.
|
| BUT there's a lot of traps like this hidden everywhere
| probably, and those will rear their ugly heads at some point.
| Wish there was a good test generation tool to go with the code
| generation tool...
| danenania wrote:
| One thing I've found in doing a lot of coding with LLMs is
| that you're often better off updating the initial prompt and
| starting fresh rather than asking for fixes.
|
| Having mistakes in context seems to 'contaminate' the results
| and you keep getting more problems even when you're
| specifically asking for a fix.
|
| It does make some sense as LLMs are generally known to
| respond much better to positive examples than negative
| examples. If an LLM sees the wrong way, it can't help being
| influenced by it, even if your prompt says very sternly _not_
| to do it that way. So you 're usually better off re-framing
| what you want in positive terms.
|
| I actually built an AI coding tool to help enable the
| workflow of backing up and re-prompting:
| https://github.com/plandex-ai/plandex
| withinboredom wrote:
| As someone who uses LLMs on my hobby projects to write
| code, I've found the opposite. I usually fix the code, then
| send it in saying it is a refactor to clarify things. It
| seems to work well enough. If it is rather complex, I will
| paste the broken code into another conversation and ask it
| to refactor/explain what is going on.
| berniedurfee wrote:
| I burned a couple hours debugging some generated code only to
| finally realize copilot was referencing a variable as
| 'variableO1'.
|
| Artificial Incompetence indeed!
| bckr wrote:
| > Using code from a generative AI without understanding what it
| does is never a good idea.
|
| Yes.
|
| AI as a faster way to type: Great!
|
| AI as a way to discover capabilities: OK.
|
| Faster way to think and solve problems: Actively harmful.
| sholladay wrote:
| So have an AI do it for you!
| jowdones wrote:
| Retail "traders" are the textbook definition of mentally
| challenged obnoxiousnes. Go meet them on forums like
| EliteTrader.com and you will soon realize who you are dealing
| with.
|
| It's your fault really. You don't build custom software for guys
| having the intellectual capacity and budget of a tractor driver
| unless you enjoy pain.
| wh-uws wrote:
| I will take all of these customers. Please forward them to me.
| atomic128 wrote:
| For real: profitable consulting businesses have been formed to
| help LLM programmers. BUGFIX 66, for example, and various
| others. They can charge substantial money to help customers
| cross that "last mile" and get their LLM-generated pile of code
| working.
| schainks wrote:
| I am literally doing this right now for a client with a
| prototype, never realized the market is as big as people are
| saying!
| cbg0 wrote:
| These "customers" were receiving free help. If they wanted to
| spend money on quality software they wouldn't have used an LLM.
| gumby wrote:
| > Helping a customer solve challenges is often super rewarding,
| but only when I can remove roadblocks for customers who can do
| most of the work themselves.
|
| One thing I loved about doing technical enterprise sales is that
| I'd meet people doing something I knew little or nothing about
| and who didn't really understand what we offered but had a
| problem they could explain and our offering could help with.
|
| They'd have deep technical knowledge of their domain and we had
| the same in ours, and there was just enough shared knowledge at
| the interface between the two that we could have fun and useful
| discussions. Lots of mutual respect. I've always enjoyed working
| with smart people even when I don't really understand what they
| do.
|
| Of course there were also idiots, but generally they weren't
| interested in paying what we charged, so that was OK.
|
| > Helping a customer solve challenges is often super rewarding,
| but only when I can remove roadblocks for customers who can do
| most of the work themselves.
|
| So I feel a lot of sympathy for the author -- that would be
| terribly soul sucking.
|
| I guess generative grammars have increased the number of "I have
| a great idea for a technical business, I just need a technical co
| founder" who think that an idea is 90% of it and have no idea
| what technical work actually is.
| cl3misch wrote:
| Your second quote is the same as the first one. Did you copy
| the same one twice by accident?
| underdeserver wrote:
| I suspect the quote was pasted by mistake the first time.
| gumby wrote:
| Yeah, I moved the paragraphs around and pasted the quote in
| where it belonged, forgetting that it had been pasted at
| the top. Too late to edit, though.
| alex-moon wrote:
| This is honestly something I'm grateful for a lot of the time.
| I'm presently running a tech start-up in a highly technical
| domain (housebuilding, in a word) which also happens to be
| pretty hostile to businesses. People look at a planning
| application like "Why are there hundreds of documents here?"
| and it's because yeah, it is hard - there are huge numbers of
| variables to take into account, and the real "art" of urban
| design is solving for all of them at once. Then you send it to
| planning and basically no-one is happy, why haven't you done
| this and what are you going to do about that. You have to be
| pretty creative to survive.
|
| Before that, I worked in a digital print organisation with a
| factory site. This factory did huge volumes on a daily basis.
| It was full of machines. They had built up a tech base over
| years, decades, and it was hyper-optimised - woe betide any dev
| who walked into the factory thinking they could see an
| inefficiency that could be refactored out. It happened multiple
| times - quite a few devs, myself included, learned this lesson
| the hard way - on rare occasion thousands of lines of code had
| to be thrown out because the devs hadn't run it past the
| factory first.
|
| It's an experience I'd recommend to any dev - build software
| for people who are not just "users" of technology but builders
| themselves. It's not as "sexy" as building consumer-facing
| tech, but it is so much more rewarding.
| sproosemoose wrote:
| The crypto product in question is for a very young age group.
|
| https://pump.fun/board
| userbinator wrote:
| Cryptocurrent trading tools? The susceptibility of people to get-
| rich-quick scams and the desire to not do even the minimum of
| work is surely correlated.
|
| Stop poisoning the well and then complaining that you have to
| drink from it.
| tdignan wrote:
| OP should set up an AI chatbot to triage his customer support. It
| probably wouldn't be that hard to send that code right back to
| GPT and get a fix suggestion to the customer instantly. Stick
| your documentation for each endpoint in a vector database, and
| use RAG to give them a quick fix. If it doesn't work let them go
| to level 2 support _for a fee_.
| thomasahle wrote:
| These kind of support requests are also a big issue for open
| source projects.
|
| Hanging out at the DSPy discord, we get a lot of people who
| need help with "their" code. The code often uses hallucinated
| API calls, but they insist it "should work" and our library
| simply is buggy.
| bruce511 wrote:
| A tiered approach to sales can help here.
|
| Cheap version offers minimal support. (Although you still have to
| sift "bug reports" into my problem/ your problem.)
|
| Standard version allows for more support (but still rate
| limited.)
|
| Developer version charges for time spent, in advance.
|
| This helps because people only expect free support if you don't
| explicitly offer something else. If you offer paid support them
| it's reasonable and expected that there are limits on free
| support.
| gorbachev wrote:
| I've been saying for a while now that there's an absolute gold
| mine waiting for people who want to specialize in fixing AI
| generated applications.
|
| A lot of businesses are going to either think they can have
| generative AI create all of their apps with the help of a cousin
| of the wife of the accountant, or they unknowingly contract a 10x
| developer from Upwork or alike who uses generative AI to create
| everything. Once they realize how well that's working out, the
| smart ones will start from scratch, the not-so-smart will attempt
| to fix it.
|
| Develop a reputation for yourself for getting companies out of
| that pickle, and you can probably retire early.
| PeterStuer wrote:
| Doesn't solve the problem that the budget they has in mind for
| the app was $300, while an experienced dev can directly see
| this is going to be a $20K v.1 before change requests project.
| KoolKat23 wrote:
| And this is why people continue to do it. And at that, with
| such a discrepancy, why not? Proof of concept, you know if it
| adds value and makes money, enough money to pay $20k.
| urbandw311er wrote:
| I think the missing proof point is whether companies would
| fork out the extra $19,700 once they understand the actual
| cost.
| gorbachev wrote:
| So they'll try 5 $300 fixes, and then either give up
| entirely, or figure out that maybe the 100 developers they
| ignored who told them it's gonna cost $20K were right.
| paretoer wrote:
| Maybe for a very short amount of time.
|
| I suspect this quickly will be like specializing in the repair
| of cheap, Chinese made desk lamps from Walmart.
|
| If the cheap desk lamp breaks, you don't pay someone to fix it.
| You buy another one and often it will be a newer , better
| model. That is the value proposition.
|
| Of course, the hand crafted, high end desk lamp will be better
| but if you just want some light, for many use cases, the cheap
| option will be good enough.
| djeastm wrote:
| >I've been saying for a while now that there's an absolute gold
| mine waiting for people who want to specialize in fixing AI
| generated applications.
|
| The real savvy ones will use later generations of LLMs to fix
| the output of early ones
| nbzso wrote:
| This is only the beginning. Imagine this when AI bot chains and
| "agents" replace conveniently junior devs on scale. Someone will
| "hack" an API/Lang Chain/Insert-LLM-Framework Solution.
|
| The next decade of support business is here. Fix my
| "hallucination" market. Thank you, Microsoft. You did it again.
| delifue wrote:
| Someone mentioned "hallucination-based API design" on twitter (I
| cannot find it now). It's designing API by LLM hallucination. If
| there is a common hallucination API call, just add that API. This
| will make the API more "friendly" and resemble common similar
| APIs.
|
| Considering that LLM can hallucinate in different ways
| unpredictably, not sure whether it will work in practice.
| sim7c00 wrote:
| I doubt LLM hallucinations will produce good secure code.
|
| In my opinion, using code from LLMs, which might see your
| program come to life a bit quicker, will only enhance the time
| needed for debugging and testing, as there might be bugs and
| problems in there ranging from trivial things (unused
| variables) to very subtle and hard to find logic issues which
| require a deeper knowledge of the libraries and frameworks
| cobbled together by an LLM.
|
| Additionally, it takes out a lot of the knowledge of these
| things in the long run, so people will find it more and more
| challenging to properly do this testing and debugging phase.
| IanCal wrote:
| The description here isn't about using LLM hallucinations as
| code. It's using them as example API users and seeing what
| they get stuck on.
|
| Let's say you've got some rest API. Are they trying to send
| PATCH requests but you only support POST? Do they keep trying
| to add pagination or look for a cursor? Do they expect
| /things to return everything and /things/id to return a
| single one but you have /things and /thing/1 ?
|
| None of this is to say you must do whatever they are trying
| to do but it's a very cheap way of seeing what introducing
| your API/library to a new user might be like. And, moreover,
| you can only ever introduce someone once but you can go again
| and again with a LLM. Review the things they're doing and see
| if actually you've deviated from broad principles and if a
| change/new endpoint actually would make things much easier.
|
| Fundamentally you can break this down to "if there's someone
| who can do some coding but isn't a top class programmer, how
| much do I need to explain it to them before they can solve a
| particular problem?"
| sim7c00 wrote:
| Ah, sorry, totally didn't get that it was using them as
| users of the API.
|
| I would recommend API fuzzers here, which cycle through all
| available possibilities, and generate 'garbage' inputs as
| well as structured inputs regarding the specifications the
| APIs implement (http, and the underlying structures
| provided to the api endpoints).
|
| an LLM would likely not perform an exhaustive test.
|
| There are several projects freely available to run such
| fuzz tests. Eventhough that does eat up considerable
| resources, and 'resetting' state during such tests can be
| problematic at best, the same would apply for LLM based
| testing. (database gets screwed in some way, and the next
| request is not on a 'clean' state.)
| stavros wrote:
| > Ah, sorry, totally didn't get that it was using them as
| users of the API.
|
| Originally, it wasn't. From the GP post:
|
| > If there is a common hallucination API call, just add
| that API. This will make the API more "friendly" and
| resemble common similar APIs.
|
| If it's just users, you wouldn't be adding the APIs.
| IanCal wrote:
| Users were calling their API.
| sim7c00 wrote:
| programs call apis.
| IanCal wrote:
| This is the silliest argument
|
| I write API calls. I am a user of apis. If you want to be
| more pedantic than this I simply do not care and you
| should evaluate what benefit you are bringing to this
| conversation.
| IanCal wrote:
| I think you're picturing this differently again.
|
| It's not testing the API for bugs, it's asking whether
| the API you have follows typical norms and isn't missing
| things. Is your API _confusing_ or _incomplete_?
|
| Imagine you have a library dealing with chat rooms, and
| LLMs keep trying to call "room.getUser(id)" and
| "room.getUsers()" when your API is actually
| "room.user(filter:{id:id})[0]" and
| "room.user(filter:None)". Maybe that's a sign that your
| API, while _totally functional_ is going to be confusing
| to other programmers. Maybe you don 't have anything that
| lists all current users, or there's a flag or option
| somewhere that would make sense or be more typical.
|
| A fuzzer won't tell you that.
|
| For the original post, you could imagine that a user
| might want the price of a token in a different currency,
| but maybe it currently returns everything in dollars. An
| LLM might try and add ?currency=GBP . Maybe an LLM keeps
| expecting to see in the response data the last updated
| timestamp to check data freshness, but it isn't something
| you add right now.
| sim7c00 wrote:
| fair point again. i dont think this is the way to go in
| either case. of your api is easy to clearly document its
| good. if you struggle with that, a programmer trying to
| call it will also. regardless of what an LLM makes of it.
|
| it seems like trying to facilitate bad things. make good
| things instead.
| IanCal wrote:
| > of your api is easy to clearly document its good
|
| If it breaks a lot of conventions or norms, no it's
| probably not good.
|
| But also the point here is *how much do you need to
| really explain?". If you have apis that require a lot of
| re-explanation to llms to force them to do the right
| thing "no getUsers doesn't exist use users(filter)
| where... YOU MUST USE users(filter)!" That's a sign your
| API might be going counter to a broad estimation of
| existing code.
|
| > it seems like trying to facilitate bad things. make
| good things instead.
|
| I have seen so many people make something that is obvious
| to them and so few others that "simply write good things
| from the start" is pointless advice.
| netcan wrote:
| Idk if "hallucination-based API design" specifically is The
| Way.
|
| There might be other ways of achieving the same goal. Also,LLM
| hallucination is changing/improving quite rapidly.
|
| That said, "Designed for LLM" is probably a very productive
| pursuit. Puts you in the right place to understand the problems
| of the day.
| port19 wrote:
| Adding a couple aliases for your endpoint might be a decent
| middle ground where you throw the hallucinating "AI" a bone,
| without contorting yourself at its will
| Flop7331 wrote:
| Let's all give ourselves extra prosthetic fingers and mutilate
| our ears while we're at it.
| RegW wrote:
| I suppose it would be relying on being trained on code that
| followed good practice. If this is true then we might suppose
| that this API isn't following good practice. However, a
| gigantic feedback loop is appearing on the horizon.
|
| The AI of tomorrow will be trained on the output of AI today.
|
| (Somewhere in my memory, I hear an ex-boss saying "Well that's
| good - isn't it?")
| Raicuparta wrote:
| You're going at it all wrong. You just need another LLM on the
| other end of the API too.
| Pikamander2 wrote:
| > Often this takes the form of trying to access an endpoint that
| does not exist, or read a property off the API response that does
| not exist. After probing a bit more, my suspicions are usually
| confirmed -- ChatGPT hallucinated that endpoint or property
|
| In some cases, you might be able to use this to your advantage to
| improve the product.
|
| When working with third-party APIs, I've often run into
| situations where my code could be simplified greatly if the API
| had an extra endpoint or parameter to filter certain data, only
| to be disappointed when it turns out to have nothing of the sort.
|
| It's possible that ChatGPT is "thinking" the same thing here;
| that most APIs have an X endpoint to make a task easier, so
| surely yours does too?
|
| Over time I've sent in a few support tickets with ideas for new
| endpoints/parameters and on one occasion had the developer add
| them, which was a great feeling and allowed me to write cleaner
| code and make fewer redundant API calls.
| ben_w wrote:
| > It's possible that ChatGPT is "thinking" the same thing here;
| that most APIs have an X endpoint to make a task easier, so
| surely yours does too?
|
| While this is possible, I would caution with an anecdote from
| my ongoing side project of "can I use it to make browser
| games?", in which 3.5 would create a reasonable Vector2D class
| and then get confused and try to call .mul() and .sub() instead
| of the .multiply() and .subtract() that it had just created.
|
| Sometimes it's exceptionally insightful, other times it needs
| to RTFM.
| fenomas wrote:
| I feel like we'll eventually all agree that it's a mistake to
| ask a generalist LLM for code. I've found ChatGPT to be fine
| at talking _about_ code - like describing the difference
| between two APIs - but for generating nontrivial chunks of
| working code I think it 's miles behind Copilot and similar.
|
| And I assume that's just because, y'know, ChatGPT can write
| sonnets and translate Korean to Swahili and whatnot. It's
| amazingly broad, but it's not the right tool for the
| comparatively narrow problem of code generation.
| Kuinox wrote:
| Copilot uses ChatGPT.
| fenomas wrote:
| It's powered by the same models, but it's not submitting
| questions to the Q&A prompt like people do when they ask
| ChatGPT to generate code for them.
|
| (..I guess? I don't think any of it is public - one might
| naively suppose that by now it's actually using only a
| subset of ChatGPT's MoEs, or something, but who knows?)
| t_mann wrote:
| another consideration: if a popular AI model hallucinates an
| endpoint for your API for one customer, chances are another
| customer will run into the same situation
| jonplackett wrote:
| I would suggest adding a help section with advice for people
| using ChatGPT that sets expectations and also gives a pre-written
| prompt for them to use.
|
| Something like.
|
| Some of you may use ChatGPT/Copilot to help with your coding.
| Please understand this service is designed for professional
| programmers and we cannot provide extensive support for basic
| coding issues, or code your app for you.
|
| However if you do want to use ChatGPT here is a useful starting
| prompt to help prevent hallucinations - though they still could
| happen.
|
| Prompt:
|
| I need you to generate code in ____ language using only the
| endpoints described below
|
| [describe the endpoints and include all your docs]
|
| Do not use any additional variables or endpoints. Only use these
| exactly as described. Do not create any new endpoints or assume
| any other variables exist. This is very important.
|
| Then give it some examples in in curl / fetch / axios / python /
| etc.
|
| Maybe also add some instructions to separate out code into
| multiple files / endpoints. ChatGPT loves to make one humungous
| file that's really hard to debug.
|
| ChatGPT works fairly well if you know how to use it correctly. I
| would defo not trust it with my crypto though, but I guess if
| some people wanna that's up to them. May as well try and help
| them!
| markatkinson wrote:
| I'm tired of fixing my own AI generated code.
| xg15 wrote:
| We have come full circle: Silicon Valley is finally disrupting
| the software engineering industry...
| orbit7 wrote:
| As a basic offering have good documentation and a community
| support forum and the option to report bugs.
|
| Make the type of support you are providing paid, this could be in
| tiers.
|
| Outsource support as needed.
| amai wrote:
| I'm tired of fixing my colleagues' AI generated code. They churn
| out a lot of code in a short amount of time, but then we loose
| the saved time again because during pull request review they
| often cannot explain what this code is actually doing. Maybe I
| should use an AI for code review, too?
| throwuxiytayq wrote:
| Why are these people employed? Isn't that a bit like a fake
| employee who outsources his work behind your back? You can't
| work with the guy because he literally doesn't even know or
| understand the code he's pushing to the repo
| ramesh31 wrote:
| >Why are these people employed?
|
| After being stuck on a project with a few of them recently,
| I've started to figure it out.
|
| They know they are incompetent, and are afraid of being
| around competent people who would call them out. So they link
| up with other likewise incompetents to maintain a diffusion
| of responsibility, so the finger can't be pointed at any one
| person. And when the system they are building inevitably
| fails horribly, management can't really fire the whole team.
| So they bumble along and create "make work" tickets to fill
| time and look like something is happening, or they spend
| their time "fixing" things that should have never existed in
| the first place by layering mountains of hacks on top, rather
| than reassessing anything or asking for help. Rinse and
| repeat once the project has clearly failed or been abandoned.
| j-a-a-p wrote:
| > My API is just a few well documented endpoints. If you can
| figure out how to send a POST request using any programming
| language, you should have no problem using it. But that seems to
| be too high a bar for the new generation of prompt-engineer
| coders.
|
| Nice. The time has come that we need to design the API with the
| LLM in mind. Or, to rephrase that, test your API that it is
| working with the popular LLM tools regularly.
| OJFord wrote:
| I think this is the point to establish a 'community', and in
| particular to only 'offer' community support (or charge more for
| spending your own time on it).
|
| Some people will enthusiastically fix other customers' generated
| crap for free, let them.
| treprinum wrote:
| Set firm boundaries and reject/ghost customers that want to build
| their app for free and are "angry". Those never lead to anything
| profitable.
| tpoacher wrote:
| Same, but for academic documents.
|
| It used to be that a 'bad' document had things 'missing', which
| were easy to spot and rephrase / suggest improvements.
|
| Now a 'bad' document is 50 pages of confusing waffle instead of
| 5, you need to get through the headache of reading all 50 to
| figure out what 'sentence' the person was trying to express at
| the place where an autogenerated section has appeared, figure out
| if the condensed document is something that actually had any
| merit to begin with, THEN identify what's missing, and then ask
| them to try again.
|
| At which point you get a NEW 50-page autogenerated document and
| start the process again.
|
| F this shit.
| shireboy wrote:
| I can empathize, but am also wondering if some of these are
| feature requests in disguise. For "how to call api" docs, sample
| code, and even client libraries can be generated from your
| OpenAPI specs. Link to the docs in every reply. The more complex
| asks could be translated "build this endpoint and charge me for
| it". If all else fails, set up partnership with devs who want to
| fix/build customers crap and figure out some copy to direct the
| more complex asks to them.
| jollyllama wrote:
| Agreed, client libs and sample applications have fallen by the
| wayside and could provide a solution here. It makes it very
| obvious to the customer when they download and run something
| that works, and then their changes break it, that the issue is
| with their changes.
| afro88 wrote:
| > Often this takes the form of trying to access an endpoint that
| does not exist, or read a property off the API response that does
| not exist. After probing a bit more, my suspicions are usually
| confirmed -- ChatGPT hallucinated that endpoint or property
|
| This is an opportunity. Add another tier to your pricing
| structure that provides an AI that assists with coding the API
| connection. Really simple Llama 3.1 that RAGs your docs. Or
| perhaps your docs fit in the context already if the API is as
| simple as it sounds.
| netcan wrote:
| This is totally understandable, valid, etc.
|
| OTOH, see script kiddies, WYSIWYG Stack Overflow C&P, etc.
|
| It's just the way things are now.
| paxys wrote:
| > The worst is when a request starts out simple -- I help them
| fix one hallucination -- but then that customer wants to build
| more complex logic, and somehow I've set the expectation that I
| will provide unlimited free support forever. I've gotten a number
| of angry messages from customers who essentially want me to build
| their whole app for free.
|
| Welcome to the life of any consultant/support agent/open source
| maintainer ever. AI isn't the problem here, managing expectations
| is.
| paxys wrote:
| I can guarantee that all these users are following some "hustle
| university" course peddled by a Twitter influencer. Crypto and AI
| are the two favorite words of all these get rich quick scams.
| pvillano wrote:
| I won't say all investors are entitled and overconfident,
| inspired by grifters, emboldened by survivorship bias, and
| motivated by greed. That would be rude
| bbarnett wrote:
| You can be a Google dev, and make half a million a year! For
| only $29.95, we'll show you how to empower yourself with the
| wonders of AI!
| Tade0 wrote:
| Typically the decimal separator is two steps to the right
| here.
|
| The other day my friend showed me a curious screenshot where
| an, ahem, dev influencer showed all the courses he's done.
|
| Problem is, just one of those six courses amounted to two
| average net salaries in the region and they were fairly
| basic.
|
| Either the guy in question had a year's worth of runway (then
| why even bother getting into IT?), got into debt or... didn't
| actually pay for any of this and it was not disclosed.
|
| I do my best to dissuade people from spending their hard
| earned money on such things, but a significant chunk
| unfortunately does not listen.
| mikewarot wrote:
| I'm on the other side of this when it comes to the C programming
| language. I've avoided it for decades, preferring Pascal, or even
| Visual Basic.
|
| The single best thing to help someone in my place is clear and
| coherent documentation with working examples. It's how I learned
| to use Turbo Pascal so long ago, and generally the quickest way
| to get up to speed. It's also my biggest gripe with Free
| Pascal... their old help processing infrastructure binds them to
| horrible automatically generated naming of parameters as
| documentation, and nothing more.
|
| Fortunately, CoPilot doesn't get impatient, and I know this, so I
| can just keep pounding away at things until I get something close
| to what I want, in spite of myself. ;-)
| linsomniac wrote:
| Seems like the opportunity here is for the author to sell an AI
| that is trained on the API and documentation.
| andai wrote:
| As the author says, the errors are very easy to fix. Easy enough
| for GPT! He should set up a support chatbot. Only seems fair? ;)
|
| I'm half joking, but in most cases I found that GPT was able to
| fix its own code. So this would reduce support burden by like
| 90%.
|
| I hate chatbot support as much as the next guy, but the
| alternative (hiring a programmer to work as a customer support
| agent on AI generated code all day?) sounds not only cruel, but
| just bad engineering.
| dimal wrote:
| I've found that it's often able to fix its own code when I'm
| able to understand that problem and state it clearly. On its
| own, it tends to just go in circles and proudly declare that
| its solve the problem, even though it hasn't. It needs a
| knowledgeable person to guide it.
| cynicalpeace wrote:
| AI is going to be like any other tool. If you don't know how to
| use it, you may end up hurting yourself.
|
| If you know how to use it, it will make you 100x more efficient.
| pacoWebConsult wrote:
| Seems to me like you have an opportunity to develop a couple SDKs
| in your customers' favorite languages (probably python and
| typescript) and a simple "Get Started" template that could
| alleviate a lot of these requests. Show them the base case, let
| them figure out how to work with your api via an SDK instead of
| directly with an API and let the advanced users build their own
| SDKs in whatever language they prefer. Since its, as OP claims, a
| simple HTTP API, the SDK could be generated with OpenAPI client
| generation tools.
| jdance wrote:
| Seems like a good way to have a bunch of new products to also
| support
| pacoWebConsult wrote:
| Maintaining an OpenAPI spec when you make changes to your API
| and regenerating the client SDKs through CI is really not a
| ton of extra work. A template to show a dead simple usage of
| your API can pay dividends as it lowers the barrier to a
| customer adopting your product over a competitor's.
| maxidorius wrote:
| And then the customers will open support requests for code
| generated by an AI that misuse that very SDK. It doesn't look
| like OP's issue is with the code per say, only with the lack of
| skills of its customers, _regardless_ of the code they write...
| dpassens wrote:
| I don't really know how to point this out without sounding
| rude and obnoxious, which is not my intention, but it's "per
| se" (Latin for by itself), not "per say".
| pacoWebConsult wrote:
| At the very least, GitHub Copilot will have an easier time
| with an SDK loaded into context than an API documented only
| on the web. If the customers are using typescript then
| they'll have some red squiggles that at least some of these
| people will bother to read prior to asking for help. The
| uninformed consumer of OPs API will probably be more
| comfortable to work with an SDK instead of writing their own
| clients.
|
| The way I see it is that OP can either complain about
| customers being annoying, which will happen whether or not OP
| does anything about his problem, or OP could proactively
| produce something to make their product better for the
| demographic they're targeting. At this point it's pretty
| clear that the users would rather be helped than help
| themselves, so meeting them where they're at is going to be
| more productive than trying to educate them on why they suck
| at both coding and asking for help (or free work).
| snakeyjake wrote:
| AI, Rust, crypto, Medium, SAAS startups
|
| It's a fuckin perfect hacker news bingo.
|
| Magnificent!
| pmarreck wrote:
| The game might change (a little) when it can actually run the
| code it generates, examine any errors and potentially fix based
| on that.
|
| And when you use a language that has better checks all around.
| suprt411 wrote:
| Do you want to outsource this L1 L2 or even L3+ support? Lets
| talk
| bkazez wrote:
| Charge for engineering support and hire someone to do it for you!
| ramesh31 wrote:
| Welcome to the hell that is being a senior dev nowadays. Every
| junior with a Copilot license all of the sudden fancies
| themselves an expert programmer now. It's absolutely maddening.
| adverbly wrote:
| Another concern is around reviewing it.
|
| I can't tell in a pull request what someone wrote themselves, or
| to what level of detail they have pre-reviewed the AI code which
| is now part of the pull request before allowing it to get to me.
| I can tell you I don't want to be fixing someone else's AI
| generated bugs though... Especially given that AI writes
| more/less dry/more verbose code, and increases code churn in
| general.
| hoosieree wrote:
| Just add more AI, that'll solve everything.
|
| [edit] unfortunately I think I need to point out the above is
| sarcasm. Because there really are people using AI to review AI-
| generated code, and they _do not see the problem with this
| approach_.
| jrochkind1 wrote:
| I am no partiuclar fan of AI (and actually haven't used copilot
| or similar myself ever), but the real problem here is not AI but:
|
| > I've gotten a number of angry messages from customers who
| essentially want me to build their whole app for free.
|
| I guess AI maybe encourages more people to think they can build
| something without knowing what they are doing. i can believe that
| for sure. There was plenty of that before AI too. Must be even
| worse now.
| lacoolj wrote:
| lol this is just like trying to help people in programming
| discords that are literally using AI on screen to write and
| rewrite the entire app as they go. then they run into an issue,
| ask for help, and don't understand when you say "there's a memory
| leak 50 lines down" or "you have to define that variable first".
|
| AI is a great tool to help someone start an idea. When it goes
| past that, please don't ask us for help until you know what the
| code is doing that you just generated.
| zzz95 wrote:
| Every problem hides opportunities! I can see a future where
| documentation will be replaced by a plugin to an AI code service.
| Instead of providing users with documentation on how to use the
| package, devs will be training an LLM on how to assist the user
| in generating the interface code. An elaborate Chat GPT prompt
| for instance.
___________________________________________________________________
(page generated 2024-08-22 17:01 UTC)