[HN Gopher] Show HN: How to use ChatGPT+ARKit to script experien...
       ___________________________________________________________________
        
       Show HN: How to use ChatGPT+ARKit to script experiences with
       natural language
        
       Video demonstration: https://youtu.be/bJFa5HOmhl4  I created a demo
       involving ChatGPT, OpenAI's Whisper running on-device for speech-
       to-text (because why not?), and ARKit. By defining a custom
       JavaScript environment with JavaScriptCore and describing it to
       ChatGPT, I was able to get it to produce functioning scripts that
       instantiate and manipulate 3D objects.  Sketchfab is used to import
       3D assets on the fly. Google Poly had a much better catalog of
       assets for this sort of thing but sadly, it's no longer available.
       The README.md describes how it all works but the gist of it is that
       user prompts are wrapped in a larger prompt that describes the
       environment (i.e., which functions are available to use and how
       they work) and constraints. The code is then executed directly.
       Assets for objects are fetched as needed by searching Sketchfab.
        
       Author : trzy
       Score  : 27 points
       Date   : 2022-12-21 18:21 UTC (4 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | photoGrant wrote:
       | This just gave me chills knowing at some point in the future,
       | Children will be able to dream up their perfect fantasy land and
       | program their surroundings by just asking for it. Then these
       | children will become adults. The future is (undefined)
        
         | trzy wrote:
         | I would love for this to become robust enough to give to kids
         | and see what they dream up. Unlike adults, kids are far less
         | inhibited at coming up with fun prompts, but that also means
         | they would break this admittedly brittle prototype :D
        
       | ladberg wrote:
       | JavaScriptCore is definitely underrated! I've used it for quickly
       | prototyping business logic of apps because you can iterate
       | without having to recompile, reload, or even reopen the app on
       | device. You can also use Safari to open up a web inspector for
       | the context in your app and do some remote debugging.
        
         | trzy wrote:
         | Question: I couldn't figure out how to export properties that
         | were writable in JS. I describe the problem in Entity.js. Any
         | pointers would be much appreciated! :)
        
       | coyotespike wrote:
       | I love this, a real future shock moment. Jarvis wen??
       | 
       | Have you seen LangChain? It has a similar idea of getting GPT to
       | output runnable code, and then using that to search the web or
       | run Python, and giving the results back to GPT.
        
         | trzy wrote:
         | Thanks for sharing that! I hadn't heard of LangChain. That
         | looks super cool. I really hope ChatGPT gets turned into a
         | proper API because fighting OpenAI's bot countermeasures is a
         | real time sink.
        
       ___________________________________________________________________
       (page generated 2022-12-21 23:01 UTC)