[HN Gopher] A web version of Anthropic's prompt engineering inte...
       ___________________________________________________________________
        
       A web version of Anthropic's prompt engineering interactive
       tutorial
        
       Author : thenameless7741
       Score  : 52 points
       Date   : 2024-05-17 23:16 UTC (23 hours ago)
        
 (HTM) web link (thenameless.net)
 (TXT) w3m dump (thenameless.net)
        
       | brylie wrote:
       | Interesting that the examples use XML for structuring/annotating
       | the prompts. Is there any available comparison of using XML,
       | JSON, or even Markdown for prompts and structured output?
       | Markdown would seem like the prompt format with the least
       | friction/verbosity, but I wonder if it would have a qualitative
       | effect on the model output.
        
         | NeatPasta wrote:
         | > Is there any available comparison of using XML, JSON, or even
         | Markdown for prompts and structured output?
         | 
         | I've done a lot of testing with this and found that XML is the
         | best input AND output if you want to produce machine readable
         | data. Markdown is okay as input, but testing shows better
         | accuracy if input components are wrapped in XML tags. While it
         | would be ideal for extracting data, JSON syntax is too
         | sensitive as output. Often an offending character will be
         | unescaped no matter how good your prompt is.
         | 
         | If you stay within the very basic syntax of XML (just tags) it
         | is also the best output. I suspect it is to do with the
         | descriptive nature of the tag structure. Something wrapped in
         | the tag <summary> will hint to the model that it should produce
         | a summary, reinforcing the input prompt as the output is being
         | constructed.
        
           | ai_what wrote:
           | This is really good to know, thanks! Was this for every LLM
           | or only Claude?
        
         | pietz wrote:
         | I think we need to differentiate between a few things:
         | 
         | 1. The specific way different models were trained.
         | 
         | Claude was specifically trained with these XML tags in the
         | input and output. For Claude it works well to say things along
         | the lines of "use a scratchpad for thinking out loud in XML
         | tags first". While you can force GPT to do the same, it may not
         | be as "natural" e.g. high chance of GPT3.5 not doing it.
         | 
         | 2. XML tags vs full XML
         | 
         | Using XML tags doesn't mean it would be amazing at outputting
         | full valid XML. XML tags are great because there's no further
         | formatting needed within. If you only want to separate sections
         | in your prompt or response, XML tags are perfect for clearly
         | marking start and end, as well as featuring syntax (<>) that's
         | rarely seen in natural text.
         | 
         | If it comes to generating a full data schema valid object, my
         | experience is that JSON works way better than XML. It just
         | seems to be a good middle ground between structure while having
         | little overhead. I remember a recent AirBnB posts saying that
         | YAML works even better than JSON. That's not my personal
         | experience. Plus, YAML is something a bit weird and slow on
         | read.
        
       ___________________________________________________________________
       (page generated 2024-05-18 23:00 UTC)