Post ASFIE6O2x3SP0OVLeq by simon@fedi.simonwillison.net
 (DIR) More posts by simon@fedi.simonwillison.net
 (DIR) Post #ASFI1BsrCdA8Hzl8yW by simon@fedi.simonwillison.net
       2023-02-01T19:42:00Z
       
       0 likes, 0 repeats
       
       I used my laptop to generate terrible Shakespeare using a custom trained language model!https://til.simonwillison.net/llms/nanogpt-shakespeare-m2I used nanoGPT by @karpathy - with some hints from issue comments I was able to train the model on my MacBook Pro M2 with a 3x performance improvement using device=mps rather than device=cpuI also built an Observable Notebook tool for plotting loss over time during training: https://observablehq.com/@simonw/plot-loss-from-nanogpt
       
 (DIR) Post #ASFIE6O2x3SP0OVLeq by simon@fedi.simonwillison.net
       2023-02-01T19:42:57Z
       
       0 likes, 0 repeats
       
       @karpathy Here's some terrible Shakespeare it wrote for me - I gave it the first three lines:GLOUCESTER:What do you think of this, my lord?KING RICHARD II:Yea, and for what to give you to jest?DUCHESS OF YORK:I amiss, I'll deny it.DUKE OF YORK:I am the matter, I beseech your graceWithalness to be your good queen;For I shall send.DUKE OF YORK:A noble lord 'gainst your father's brother's life,And that thy father is that our love I.
       
 (DIR) Post #ASFK02dIJPbNDU3liq by danhon@dan.mastohon.com
       2023-02-01T20:00:27Z
       
       0 likes, 0 repeats
       
       @simon @karpathy I was waiting for someone to show nanoGPT working with MPS; it goes back to last time I used Karpathy's rnn to generate my town names (checks) *five years ago* on my Intel MacBook Pro!
       
 (DIR) Post #ASFKEQYABXVZRsWGZ6 by karpathy@sigmoid.social
       2023-02-01T20:06:11Z
       
       0 likes, 0 repeats
       
       @simon nice! You may get even better results with the char version. The version using bpe creates a very large vocab, many many parameters , too many
       
 (DIR) Post #ASFMldTvKIc71IV0gC by zellyn@hachyderm.io
       2023-02-01T20:34:52Z
       
       0 likes, 0 repeats
       
       @simon @karpathy Interesting. That reads like something between ChatGPT and an old-school Markov Chain model…
       
 (DIR) Post #ASFTlxhSr1ajPBw3KS by springlite@fosstodon.org
       2023-02-01T21:53:22Z
       
       0 likes, 0 repeats
       
       @simon @karpathy Students in the future:  "Shakespeare wasn't all that great.  I've read his complete works on my phone and it's doggerel."