https://github.com/microsoft/visual-chatgpt

Skip to content Toggle navigation
 
Sign up

  * Product
      +  
        Actions
        Automate any workflow
      +  
        Packages
        Host and manage packages
      +  
        Security
        Find and fix vulnerabilities
      +  
        Codespaces
        Instant dev environments
      +  
        Copilot
        Write better code with AI
      +  
        Code review
        Manage code changes
      +  
        Issues
        Plan and track work
      +  
        Discussions
        Collaborate outside of code
      + Explore
      + All features
      + Documentation
      + GitHub Skills
      + Blog
  * Solutions
      + For
      + Enterprise
      + Teams
      + Startups
      + Education
      + By Solution
      + CI/CD & Automation
      + DevOps
      + DevSecOps
      + Case Studies
      + Customer Stories
      + Resources
  * Open Source
      +  
        GitHub Sponsors
        Fund open source developers
      +  
        The ReadME Project
        GitHub community articles
      + Repositories
      + Topics
      + Trending
      + Collections
  * Pricing

[                    ] 

  *  
    #
    In this repository All GitHub |
    Jump to |

  * No suggested jump to results

  *  
    #
    In this repository All GitHub |
    Jump to |
  *  
    #
    In this organization All GitHub |
    Jump to |
  *  
    #
    In this repository All GitHub |
    Jump to |

Sign in
Sign up
{{ message }}
microsoft / visual-chatgpt Public

  * Notifications
  * Fork 281
  * Star 5.7k

VisualChatGPT

License

MIT license
5.7k stars 281 forks
Star
Notifications

  * Code
  * Issues 25
  * Pull requests 7
  * Actions
  * Projects 0
  * Security
  * Insights

More

  * Code
  * Issues
  * Pull requests
  * Actions
  * Projects
  * Security
  * Insights

microsoft/visual-chatgpt

This commit does not belong to any branch on this repository, and may
belong to a fork outside of the repository.
main
Switch branches/tags
[                    ]
Branches Tags
Could not load branches
Nothing to show
{{ refName }} default View all branches
Could not load tags
Nothing to show
{{ refName }} default
View all tags

Name already in use

A tag already exists with the provided branch name. Many Git commands
accept both tag and branch names, so creating this branch may cause
unexpected behavior. Are you sure you want to create this branch?
Cancel Create
1 branch 0 tags
Code

  * Local
  * Codespaces

  *  
    Clone
    HTTPS GitHub CLI
    [https://github.com/m]

    Use Git or checkout with SVN using the web URL.

    [gh repo clone micros]

    Work fast with our official CLI. Learn more.

  * Open with GitHub Desktop
  * Download ZIP

Sign In Required

Please sign in to use Codespaces.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching Xcode

If nothing happens, download Xcode and try again.

Launching Visual Studio Code

Your codespace will open once ready.

There was a problem preparing your codespace, please try again.

Latest commit

@Moymix
Moymix Create LICENSE.md
...
865db60 Mar 10, 2023
Create LICENSE.md
865db60

Git stats

  * 19 commits

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
assets
update demo
March 9, 2023 15:45
LICENSE.md
Create LICENSE.md
March 11, 2023 00:09
README.md
Merge branch 'main' of https://github.com/microsoft/visual-chatgpt
March 10, 2023 17:06
download.sh
v0.1
March 9, 2023 11:49
requirement.txt
First Commit
March 9, 2023 11:42
visual_chatgpt.py
fix typos in the description
March 9, 2023 19:53
View code
Visual ChatGPT Demo System Architecture Quick Start GPU memory usage
Acknowledgement

README.md

 Visual ChatGPT

Visual ChatGPT connects ChatGPT and a series of Visual Foundation
Models to enable sending and receiving images during chatting.

See our paper: Visual ChatGPT: Talking, Drawing and Editing with
Visual Foundation Models

 Demo

[demo_short]

 System Architecture

                                Logo

 Quick Start

# create a new environment
conda create -n visgpt python=3.8

# activate the new environment
conda activate visgpt

#  prepare the basic environments
pip install -r requirement.txt

# download the visual foundation models
bash download.sh

# prepare your private openAI private key
export OPENAI_API_KEY={Your_Private_Openai_Key}

# create a folder to save images
mkdir ./image

# Start Visual ChatGPT !
python visual_chatgpt.py

 GPU memory usage

Here we list the GPU memory usage of each visual foundation model,
one can modify self.tools with fewer visual foundation models to save
your GPU memory:

Foundation Model Memory Usage (MB)
ImageEditing     6667
ImageCaption     1755
T2I              6677
canny2image      5540
line2image       6679
hed2image        6679
scribble2image   6679
pose2image       6681
BLIPVQA          2709
seg2image        5540
depth2image      6677
normal2image     3974
InstructPix2Pix  2795

 Acknowledgement

We appreciate the open source of the following projects:

Hugging Face   LangChain   Stable Diffusion   ControlNet  
InstructPix2Pix   CLIPSeg   BLIP  

About

VisualChatGPT

Resources

Readme

License

MIT license

Code of conduct

Code of conduct

Security policy

Security policy

Stars

5.7k stars

Watchers

52 watching

Forks

281 forks

Releases

No releases published

Packages 0

No packages published

Contributors 8

  * @Moymix
  * @CODINNLG
  * @microsoftopensource
  * @ShengmingYin01
  * @Wang-Xiaodong1899
  * @WeizhenQ
  * @bojieli
  * @heysujal

Languages

  * Python 98.4%
  * Shell 1.6%

Footer

 (c) 2023 GitHub, Inc.

Footer navigation

  * Terms
  * Privacy
  * Security
  * Status
  * Docs
  * Contact GitHub
  * Pricing
  * API
  * Training
  * Blog
  * About

You can't perform that action at this time.
You signed in with another tab or window. Reload to refresh your
session. You signed out in another tab or window. Reload to refresh
your session.