[HN Gopher] The architecture of today's LLM applications
       ___________________________________________________________________
        
       The architecture of today's LLM applications
        
       Author : kiyanwang
       Score  : 137 points
       Date   : 2023-11-19 12:41 UTC (10 hours ago)
        
 (HTM) web link (github.blog)
 (TXT) w3m dump (github.blog)
        
       | FrustratedMonky wrote:
       | Nice.
       | 
       | Any links to the actual construction of the LLM? I'm told the
       | underlying code isn't all that complicated. Hence a lot of people
       | here posting that 'everyone' will catch up to OpenAI eventually.
       | 
       | Edit: Guess I hadn't read far enough.
       | 
       | Links in the parent to some LLM's with source available.
       | 
       | https://github.blog/2023-10-05-a-developers-guide-to-open-so...
        
       | insanitybit wrote:
       | Has anyone tried using one of the models for cyber-security? I'm
       | concerned that censored models are going to make that difficult,
       | since I want to describe attacking systems.
        
         | lmeyerov wrote:
         | Yes, we work with security analysts and other folks doing data-
         | intensive investigations & analysis in Louie.ai (think being
         | able to use natural language to talk to you Splunk, OpenSearch,
         | Databricks, etc. logs/news/.., and many Python / Pandas viz +
         | wrangling + AI tools, to investigate more easily & further,
         | build autonomous hunts, smarter monitors & detections, etc)...
         | and LLMs work amazingly here.
         | 
         | Blue team is a lot easier than red team, and as part of that,
         | code tasks are easier than social engineering. It rarely comes
         | up in investigations, but you do need to set prompt persona for
         | simulating attacks.
         | 
         | At the same time, fine-tuning a model is a pretty clear way to
         | undo needing to do even that.
         | 
         | We benefit from models having business-neutral alignment by
         | default -- the typical case suffers if the default persona was
         | a racist forum troll with extreme politics. So it's more about
         | when you want to turn that off for specific tasks. It's more
         | work than we'd like, and gotchas like changing when models
         | update.
        
           | insanitybit wrote:
           | Got it, thanks. Alright that makes sense. I haven't really
           | tried much at this point, though I have had ChatGPT give me a
           | few "I can't do that" and I've had to be like "no, seriously,
           | I'm doing this because it's a _good thing_ " in the past (3.5
           | days).
           | 
           | Fine tuning does seem like a good idea.
        
         | zitterbewegung wrote:
         | I've been using a LLM to do natural language network scanning.
         | See https://www.securday.com/
         | 
         | I'm not self hosting a model yet though.
         | 
         | If they are censored I think you could find someone who has
         | gotten around it . Truly censoring a model is tough to get to
         | high parameter models that are performant.
        
       | nothrowaways wrote:
       | To make it clear, it's not about model architecture
        
       ___________________________________________________________________
       (page generated 2023-11-19 23:01 UTC)