https://github.com/01-ai/Yi-1.5 Skip to content Navigation Menu Toggle navigation Sign in * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} 01-ai / Yi-1.5 Public * Notifications * Fork 1 * Star 49 * Yi-1.5 is an upgraded version of Yi, delivering stronger performance in coding, math, reasoning, and instruction-following capability. License Apache-2.0 license 49 stars 1 fork Branches Tags Activity Star Notifications * Code * Issues 2 * Pull requests 0 * Discussions * Actions * Projects 0 * Security * Insights Additional navigation options * Code * Issues * Pull requests * Discussions * Actions * Projects * Security * Insights 01-ai/Yi-1.5 This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main BranchesTags Go to file Code Folders and files Name Name Last commit Last commit message date Latest commit History 20 Commits demo demo LICENSE LICENSE README.md README.md requirements.txt requirements.txt View all files Repository files navigation * README * Apache-2.0 license [Yi_logo_icon_light] HuggingFace * ModelScope * [?] WiseModel Discord * Twitter * WeChat Paper * FAQ * Learning Hub --------------------------------------------------------------------- * Intro * News * Quick Start * Web Demo * Deployment * Fine-tuning * API * License Intro Yi-1.5 is an upgraded version of Yi. It is continuously pre-trained on Yi with a high-quality corpus of 500B tokens and fine-tuned on 3M diverse fine-tuning samples. Compared with Yi, Yi-1.5 delivers stronger performance in coding, math, reasoning, and instruction-following capability, while still maintaining excellent capabilities in language understanding, commonsense reasoning, and reading comprehension. Yi-1.5 comes in 3 model sizes: 34B, 9B, and 6B. For model details and benchmarks, see Model Card. News * 2024-05-13: The Yi-1.5 series models are open-sourced, further improving coding, math, reasoning, and instruction-following abilities. Requirements * Make sure Python 3.10 or a later version is installed. * Set up the environment and install the required packages. pip install -r requirements.txt * Download the Yi-1.5 model from Hugging Face, ModelScope, or WiseModel. Quick Start This tutorial runs Yi-1.5-34B-Chat locally on an A800 (80G). from transformers import AutoModelForCausalLM, AutoTokenizer model_path = '' tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=False) # Since transformers 4.35.0, the GPT-Q/AWQ model can be loaded using AutoModelForCausalLM. model = AutoModelForCausalLM.from_pretrained( model_path, device_map="auto", torch_dtype='auto' ).eval() # Prompt content: "hi" messages = [ {"role": "user", "content": "hi"} ] input_ids = tokenizer.apply_chat_template(conversation=messages, tokenize=True, add_generation_prompt=True, return_tensors='pt') output_ids = model.generate(input_ids.to('cuda')) response = tokenizer.decode(output_ids[0][input_ids.shape[1]:], skip_special_tokens=True) # Model response: "Hello! How can I assist you today?" print(response) Deployment Prerequisites: Before deploying Yi-1.5 models, make sure you meet the software and hardware requirements. vLLM Prerequisites: Download the lastest version of vLLM. 1. Start the server with a chat model. python -m vllm.entrypoints.openai.api_server --model 01-ai/Yi-1.5-9B-Chat --served-model-name Yi-1.5-9B-Chat 2. Use the chat API. * HTTP curl http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "Yi-1.5-9B-Chat", "messages": [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Who won the world series in 2020?"} ] }' * Python client from openai import OpenAI # Set OpenAI's API key and API base to use vLLM's API server. openai_api_key = "EMPTY" openai_api_base = "http://localhost:8000/v1" client = OpenAI( api_key=openai_api_key, base_url=openai_api_base, ) chat_response = client.chat.completions.create( model="Yi-1.5-9B-Chat", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Tell me a joke."}, ] ) print("Chat response:", chat_response) Web Demo python demo/web_demo.py -c Fine-tuning You can use LLaMA-Factory, Swift, XTuner, and Firefly for fine-tuning. These frameworks all support fine-tuning the Yi series models. API Yi APIs are OpenAI-compatible and provided at Yi Platform. Sign up to get free tokens, and you can also pay-as-you-go at a competitive price. Additionally, Yi APIs are also deployed on Replicate and OpenRouter. License The code and weights of the Yi-1.5 series models are distributed under the Apache 2.0 license. [ Back to top [?] ] About Yi-1.5 is an upgraded version of Yi, delivering stronger performance in coding, math, reasoning, and instruction-following capability. Resources Readme License Apache-2.0 license Activity Custom properties Stars 49 stars Watchers 8 watching Forks 1 fork Report repository Releases No releases published Packages 0 No packages published Contributors 2 * @Anonymitaet Anonymitaet Yu * @Yimi81 Yimi81 Guofeng Yi Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.