Post ASRsWMBirzUPTHGNlI by ehmatthes@fosstodon.org
(DIR) More posts by ehmatthes@fosstodon.org
(DIR) Post #ASRsWIyqnOPVWUDYmW by ehmatthes@fosstodon.org
2023-02-07T21:12:03Z
0 likes, 0 repeats
I've been trying to look at ChatGPT from the perspective of people trying to learn programming. One thing seems clear so far: while it may be helpful to experienced developers, ChatGPT is not a reliable teacher.I used it to look up something relatively uncomplicated in Python: how to access the index of each item inside a loop.
(DIR) Post #ASRsWJjzy67lsiHDGq by ehmatthes@fosstodon.org
2023-02-07T21:13:38Z
0 likes, 0 repeats
The first suggestion was the most appropriate solution. That's good, but I do note it's a pretty poor example pedagogically. Learning about enumeration using a list of integers starting at 1 is a pretty confusing approach.
(DIR) Post #ASRsWKE8A4E9OANHEm by ehmatthes@fosstodon.org
2023-02-07T21:15:23Z
0 likes, 0 repeats
But then I asked it if there were other approaches, and it got much worse. It went through the standard C example. Then it showed this:It's pretty much suggesting that you use zip() to re-implement enumerate() on your own.
(DIR) Post #ASRsWKeiZDUiicoVg8 by ehmatthes@fosstodon.org
2023-02-07T21:19:06Z
0 likes, 0 repeats
And check out this monstrosity. It's using enumerate() inside a comprehension, then looping over the comprehension to pull out the results of the enumeration. 🤦♂️
(DIR) Post #ASRsWL60vjKS5HaJE0 by ehmatthes@fosstodon.org
2023-02-07T21:21:44Z
0 likes, 0 repeats
The worst part though, is when asked to evaluate the appropriateness of these solutions. ChatGPT does say that enumerate() is the best approach. But there's also this response.
(DIR) Post #ASRsWLZn919FZdW5dg by ehmatthes@fosstodon.org
2023-02-07T21:22:13Z
0 likes, 0 repeats
People new to programming, or new to a language, *should* keep their eyes out for advanced techniques, and for situations where those techniques are appropriate. But this last approach is *never* appropriate.In this case ChatGPT is not overconfident; it's too noncommittal. That's not a good thing when you're learning.
(DIR) Post #ASRsWMBirzUPTHGNlI by ehmatthes@fosstodon.org
2023-02-07T21:22:32Z
0 likes, 0 repeats
You really do need a baseline of relevant knowledge in order to make sense of the output these tools currently generate.Full article here: https://mostlypython.substack.com/p/chatgpt-is-not-a-reliable-teacher
(DIR) Post #ASRsWMaBP33Uh8hut6 by simon@fedi.simonwillison.net
2023-02-07T21:25:03Z
0 likes, 0 repeats
@ehmatthes I'm not yet convinced that this is actually a problemI've had plenty of teachers in my life who were unreliable, and who taught me things that later turned out to be misleading, or over-simplified, or just plain wrong!Learning effectively isn't about blindly trusting teachers: it's about using multiple sources of information to build your own mental model of how things work
(DIR) Post #ASRsk7exgOiQSyVHGK by simon@fedi.simonwillison.net
2023-02-07T21:26:49Z
0 likes, 0 repeats
@ehmatthes if people thing ChatGPT is infallible they're going to be in a world of pain very quicklyBut it's very, very easy to show people how unreliable it is early on in their interactions with it. Lots of people figure that out for themselves without any guidance at all!If people can learn to treat it as an unreliable teacher who has read more than them but doesn't necessarily understand any of it, I think they should be OK
(DIR) Post #ASRskH1GsouPUm0imm by simon@fedi.simonwillison.net
2023-02-07T21:27:55Z
0 likes, 0 repeats
@ehmatthes fundamentally though, this reinforces the idea for me that effectively using tools like ChatGPT isn't nearly as easy and obvious as people initially think it isTeaching people how to make the best use of these tools while avoiding their many, many failings is really important
(DIR) Post #ASRt826ry7cMCgxUW0 by ehmatthes@fosstodon.org
2023-02-07T21:34:15Z
0 likes, 0 repeats
@simon > If people can learn to treat it as an unreliable teacher who has read more than them but doesn't necessarily understand any of it, I think they should be OKThat's a really nice way to put it.I've had my share of poor teachers, and I honestly haven't gotten much from them. When I've had teachers that fit this description, I've found myself better off relying on other resources, which means I've only stuck with those bad teachers because I had to.
(DIR) Post #ASRtVOPNEInO1OkiOG by ehmatthes@fosstodon.org
2023-02-07T21:35:44Z
0 likes, 0 repeats
@simon Yes! I'm quite curious to hear stories over the coming months and years about how useful people with different backgrounds find ChatGPT and its descendants.
(DIR) Post #ASRtgbXClb15uwlHaS by simon@fedi.simonwillison.net
2023-02-07T21:35:51Z
0 likes, 0 repeats
@ehmatthes I've also tried describing them as teachers who can have a useful conversation about some topics, but can very easily start spouting increasingly ludicrous conspiracy theories of you ask them the wrong question