[HN Gopher] The code that ChatGPT can't write
___________________________________________________________________
The code that ChatGPT can't write
Author : kmdupree
Score : 12 points
Date : 2022-12-07 19:23 UTC (3 hours ago)
(HTM) web link (datachimp.app)
(TXT) w3m dump (datachimp.app)
| wilg wrote:
| I understand and agree with the point, but being able to identify
| the ambiguity of a natural language specification and ask
| clarifying questions seems well-suited to a chat-style model of
| AI interaction.
| johnthuss wrote:
| "ChatGPT helps us represent software in a programming language,
| but the hard part is representing it in English in the first
| place."
| 80x25 wrote:
| This is spot on. I'm very glad to see this post mention the "No
| Silver Bullet" essay. Despite being written in 1986, it continues
| to provide a relevant, skeptical view on the impact of these LMMs
| on software. It even has a section addressing whether AI can
| provide the "silver bullet", which is drawn upon by the OP.
|
| My immediate response to all the rightly-deserved excitement for
| ChatGPT is to reread that essay and reflect on whether this new
| tool will provide me more than marginal gains in my productivity.
| Experimenting with it a bit over the last day, my initial answer
| is, no, this doesn't provide more than marginal gains outside toy
| examples.
|
| These tools may very well factor into my workflow in the future,
| but I don't see them fundamentally changing the way I construct,
| support, debug, and maintain software.
| buffalobuffalo wrote:
| I agree with the general sentiment. I have heard more doom and
| gloom about the end of the software industry in the last week
| than I think I ever have in my career. But the question that
| keeps popping into my head is, "What would the process of laying
| off all the devs and replacing them with ChatGPT even look
| like?".
|
| And the truth is, I don't see any way it could work. Too many
| questions. Would product owners interface with the language model
| directly? Would they trust it to do devops? to debug? What would
| happen if the behavior of the system wasn't quite what they
| wanted? What if they ran into situations that required very in
| depth analysis of performance via logging? Would/Could a large
| language model orchestrate all of this?
|
| At the end of the day, what devs are paid for is their knowledge
| of a complex system. That is always going to be needed.
| MuffinFlavored wrote:
| > "What would the process of laying off all the devs and
| replacing them with ChatGPT even look like?".
|
| Why does it have to be all or none? Why can't it be incredibly
| incremental?
|
| > At the end of the day, what devs are paid for is their
| knowledge of a complex system.
|
| I've turned to ChatGPT to answer questions to things I _don 't_
| know 2-3 times this week. I can only imagine where this
| technology is headed in the next 5 years.
|
| However, you can make the argument that I knew "exactly" how to
| game it/what to ask because of my experience. I used what I
| know to ask specifically for what I don't know. :)
| Imnimo wrote:
| I would find this a lot more convincing if they had put a
| customer with an underspecified problem in front of ChatGPT and
| seen how well or poorly it could nail down the true requirements.
| I would not be shocked at all to learn that ChatGPT is not good
| at that task, but I have been surprised by its other
| capabilities, so I'm not willing to write it off without giving
| it a shot.
| kmdupree wrote:
| Hey, there! OP here. This would be a neat experiment to try.
|
| One related experiment, however, already suggests the result.
| Someone tried to get ChatGPT to solve advent of code
| challenges: https://github.com/golergka/advent-of-
| code-2022-with-chat-gp.... These challenges are very clear, and
| it already seems to struggle to get the answers. One the second
| day, it took 12 tries to get it right. If it struggles this
| much with clear requirements, I don't think it'll do well with
| vague ones.
| hmsimha wrote:
| It might come up with something "good enough" for the vague
| ones.
|
| Case in point, I got it to write a chrome extension to
| highlight new comments in Hacker News:
| https://github.com/HartS/gpt-hacker-news-extension
|
| It's not perfect, it doesn't keep track of what the user
| _actually_ saw for example. It just stores a timestamp for
| each opened thread and highlights the comments that are newer
| than that timestamp.
|
| I wouldn't have thought to solve it that way, but I
| definitely think its successor will be capable of coming up
| with "good-enough" solutions for a lot of problems humans
| currently work on.
|
| Actually, humans are often just coming up with "good-enough"
| solutions in the first place.
| catchnear4321 wrote:
| Hand-crafting new code could become a hobby activity. Most will
| simply be hand-finished. Take the crudely made starter code and
| smooth it out. Because why wait for an imperfect human to do it
| when a GPT might provide equally imperfect output?
|
| A never ending code review where the revisions become the
| responsibility of the reviewer. That won't burn anyone out.
|
| Or you get to write prompts.
___________________________________________________________________
(page generated 2022-12-07 23:02 UTC)