[HN Gopher] Show HN: finetune LLMs via the Finetuning Hub
       ___________________________________________________________________
        
       Show HN: finetune LLMs via the Finetuning Hub
        
       Hi HN community, I have been working on benchmarking publicly
       available LLMs these past couple of weeks. More precisely, I am
       interested on the finetuning piece since a lot of businesses are
       starting to entertain the idea of self-hosting LLMs trained on
       their proprietary data rather than relying on third party APIs.  To
       this point, I am tracking the following 4 pillars of evaluation
       that businesses are typically look into: - Performance - Time to
       train an LLM - Cost to train an LLM - Inference (throughput /
       latency / cost per token)  For each LLM, my aim is to benchmark
       them for popular tasks, i.e., classification and summarization.
       Moreover, I would like to compare them against each other.  So far,
       I have benchmarked Flan-T5-Large, Falcon-7B and RedPajama and have
       found them to be very efficient in low-data situations, i.e., when
       there are very few annotated samples. Llama2-7B/13B and Writer's
       Palmyra are in the pipeline.  But there's so many LLMs out there!
       In case this work interests you, would be great to join forces.
       GitHub repo attached -- feedback is always welcome :)  Happy
       hacking!
        
       Author : rsaha7
       Score  : 60 points
       Date   : 2023-09-04 15:16 UTC (7 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | axpy906 wrote:
       | Thanks for putting this project together. How does your project
       | differ from similar ones? (looked at the main repo)
        
       | kordlessagain wrote:
       | This is interesting and I want to investigate using it for
       | training templates.
       | 
       | I'm working on a couple of projects, one of which starts and
       | manages GPU backed instances on Google Cloud:
       | https://github.com/FeatureBaseDB/Laminoid
        
       | mk_stjames wrote:
       | I don't see how the loading works for the end user's custom
       | dataset. In fact, I find the layers of abstraction you have
       | between getting the finetuning dataset and the actual training
       | very opaque. I can't even tell where the dataset is coming from,
       | it doesn't appear to be an example local to this repository.
       | 
       | I think a lot of people what something like... "drop .txt files
       | of example data to train on in this /folder/ and run python
       | finetune.py /folder/
        
         | vorticalbox wrote:
         | This is actually what I was hoping for. For Web UI that you can
         | load a model then load some data and hit train.
         | 
         | You can do this in the stable defusion UI to fine tune models
         | with your own dataset
        
           | mikeravkine wrote:
           | OobaBooga supports this kind of load-and-go LORA:
           | https://github.com/oobabooga/text-generation-webui
        
       | snissn wrote:
       | Hey! I don't understand enough abt llms. Fine tuning seems like
       | something great but I feel locked out of it. I need to prepare
       | data in a question answer format? I have started to play with
       | taking things like text, articles, tweets and converting them to
       | questions but I don't think I'm doing best practices. Can you
       | help explain how to take different data sources maybe like a list
       | of documentation for an open source project and fine tune using
       | it?
        
       ___________________________________________________________________
       (page generated 2023-09-04 23:01 UTC)