https://github.com/georgian-io/LLM-Finetuning-Hub Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Customer Stories + White papers, Ebooks, Webinars + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} georgian-io / LLM-Finetuning-Hub Public * Notifications * Fork 3 * Star 124 Repository that contains LLM fine-tuning and deployment scripts along with our research findings. License Apache-2.0 license 124 stars 3 forks Activity Star Notifications * Code * Issues 0 * Pull requests 2 * Actions * Projects 0 * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Security * Insights georgian-io/LLM-Finetuning-Hub This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags Name already in use A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch? Cancel Create 5 branches 0 tags Code * Local * Codespaces * Clone HTTPS GitHub CLI [https://github.com/g] Use Git or checkout with SVN using the web URL. [gh repo clone georgi] Work fast with our official CLI. Learn more about the CLI. * Open with GitHub Desktop * Download ZIP Sign In Required Please sign in to use Codespaces. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @RohitSaha RohitSaha Merge pull request #33 from georgian-io/readme/fix_links ... e2741d9 Aug 30, 2023 Merge pull request #33 from georgian-io/readme/fix_links update links to codebase and readme e2741d9 Git stats * 108 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time assets added Red pajama August 18, 2023 16:10 bert-models updated path to images August 3, 2023 19:51 falcon update script description August 16, 2023 00:50 flan-t5 fix readme August 23, 2023 14:40 inference fix error August 18, 2023 17:02 redPajama remove redundant scripts August 28, 2023 21:08 .gitignore ignore ds_store July 24, 2023 10:28 LICENSE added license August 26, 2023 04:06 README.md update links to codebase and readme August 30, 2023 16:02 requirements.txt added tiktoken August 3, 2023 19:46 View code LLM Finetuning Hub Evaluation Framework Getting Started LLM Roadmap Contributing Correspondence README.md LLM Finetuning Hub [repo-main] LLM Finetuning Hub contains code and insights to finetune various large language models for your use-case. We stress-test both open-source and close-source LLMs through our Evaluation Framework to check their applicability for real-life business use-cases. Finetuning LLMs has never been easier. Evaluation Framework * Getting Started * LLM Roadmap * Contributing Evaluation Framework For a holistic evaluation, we will make use of the Evaluation Framework that contains 4 pillars: * [roc] Performance [roc] * [tim] Time to Train [tim] * [mon] Cost to Train [mon] * [pro] Inferencing [pro] For each of the above four pillars, we are sharing our codebase and insights to: * Assist you to leverage LLMs for your business needs and challenges * Decide which LLM suits your needs from a performance and cost perspective * Boost reproducibility efforts which are becoming increasingly difficult with LLMs We are providing scripts that are ready-to-use for: * Finetuning LLMs on your proprietary dataset via PeFT methodologies such as LoRA and Prefix Tuning * Performing hyperparameter optimization to get the maximum performance out of these models Getting Started You can start fine-tuning your choice of LLM in 4 easy steps: 1. Setup conda environment wget https://repo.anaconda.com/miniconda/Miniconda3-py38_4.11.0-Linux-x86_64.sh bash Miniconda3-py38_4.11.0-Linux-x86_64.sh source ~/.bashrc conda create --name llm_finetuning python=3.9 conda activate llm_finetuning 2. Install relevant packages git clone https://github.com/georgian-io/LLM-Finetuning-Hub.git cd LLM-Finetuning-Hub/ pip install -r requirements.txt 3. Finetune your LLM of choice For instance, to finetune Falcon-7B, do the following: cd falcon/ # navigate to Falcon folder python falcon_classification.py --lora_r 8 --epochs 5 --dropout 0.1 # finetune Falcon-7B on newsgroup classification dataset python falcon_classification_inference.py --experiment # evaluate finetuned Falcon python falcon_summarization.py --lora_r 8 --epochs 1 --dropout 0.1 # finetune Falcon-7B on samsum chat dataset python falcon_summarization_inference.py --experiment # evaluate finetuned Falcon For instance, to finetune Flan-T5-Large, do the following: cd flan-t5/ # navigate to Flan-T5 folder python flan_classification.py --peft_method prefix --prefix_tokens 20 --epochs 5 # finetune Flan-T5 on newsgroup dataset python flan_classification_inference.py --experiment # evaluate finetuned Flan-T5 python flan_summarization.py --peft_method lora --lora_r 8 --epochs 1 # finetune Flan-T5 on samsum chat dataset python flan_summarization_inference.py --experiment # evalute finetuned Flan-T5 4. Zero-shot and Few-shot your LLM of choice For instance, to use Falcon-7B on newsgroup classification task, do the following: python falcon_baseline_inference.py --task_type classification --prompt_type zero-shot python falcon_baseline_inference.py --task_type classification --prompt_type few-shot To use Falcon-7B on samsum summarization task, do the following: python falcon_baseline_inference.py --task_type summarization --prompt_type zero-shot python falcon_baseline_inference.py --task_type summarization --prompt_type few-shot NOTE: All of our experiments were conducted on the AWS EC2 instance: g5.2xlarge. It has one 24GB Nvidia GPU, and is sufficient to finetune the LLMs in this repository. LLM Roadmap Our plan is to perform these experiments on all the LLMs below. To that end, this is a tentative roadmap of the LLMs that we aim to cover, and their corresponding codebase and README links: LLM Benchmarked? Open-Source? README Codebase Flan-T5 Link Folder Falcon Link Folder RedPajama Link Folder Llama-2 OpenLlama SalesForce XGen Mosaic MPT Cerebras Writer Palmyra OpenAI GPT-3.5 Google PaLM Inflection Pi Contributing If you would like to contribute to this project, we recommend following the "fork-and-pull" Git workflow. 1. Fork the repo on GitHub 2. Clone the project to your own machine 3. Commit changes to your own branch 4. Push your work back up to your fork 5. Submit a Pull request so that we can review your changes NOTE: Be sure to merge the latest from "upstream" before making a pull request! Correspondence If you have any questions, please reach out to: * Rohit Saha (Email | LinkedIn) * Kyryl Truskovskyi (Email | LinkedIn) * Maria Ponomarenko (Email | LinkedIn) About Repository that contains LLM fine-tuning and deployment scripts along with our research findings. Topics nlp falcon classification summarization lora nlp-machine-learning fine-tuning finetuning large-language-models flan-t5 redpajama qlora llama2 Resources Readme License Apache-2.0 license Activity Stars 124 stars Watchers 4 watching Forks 3 forks Report repository Releases No releases published Packages 0 No packages published Contributors 3 * @RohitSaha RohitSaha ROHIT SAHA * @mariia-georgian mariia-georgian * @georgianpoole georgianpoole Languages * Python 66.3% * Shell 33.4% * Dockerfile 0.3% Footer (c) 2023 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time.