https://github.com/facebookresearch/sapiens Skip to content Navigation Menu Toggle navigation Sign in * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + GitHub Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions By size + Enterprise + Teams + Startups By industry + Healthcare + Financial services + Manufacturing By use case + CI/CD & Automation + DevOps + DevSecOps * Resources Topics + AI + DevOps + Security + Software Development + View all Explore + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Enterprise + Enterprise platform AI-powered developer platform Available add-ons + Advanced Security Enterprise-grade security features + GitHub Copilot Enterprise-grade AI features + Premium Support Enterprise-grade 24/7 support * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up Reseting focus You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} facebookresearch / sapiens Public * Notifications You must be signed in to change notification settings * Fork 13 * Star 208 High-resolution models for human tasks. about.meta.com/realitylabs/codecavatars/sapiens/ License View license 208 stars 13 forks Branches Tags Activity Star Notifications You must be signed in to change notification settings * Code * Issues 0 * Pull requests 0 * Actions * Projects 0 * Security * Insights Additional navigation options * Code * Issues * Pull requests * Actions * Projects * Security * Insights facebookresearch/sapiens This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main BranchesTags Go to file Code Folders and files Name Name Last commit Last commit message date Latest commit History 1 Commit _install _install assets assets cv cv det det docs docs engine engine lite lite pose pose pretrain pretrain seg seg .gitignore .gitignore CODE_OF_CONDUCT.md CODE_OF_CONDUCT.md CONTRIBUTING.md CONTRIBUTING.md LICENSE LICENSE README.md README.md git_script.sh git_script.sh View all files Repository files navigation * README * Code of conduct * License * Security Sapiens Foundation for Human Vision Models Rawal Khirodkar * Timur Bagautdinov * Julieta Martinez * Su Zhaoen * Austin James Peter Selednik . Stuart Anderson . Shunsuke Saito ECCV 2024 (Oral) Paper PDF Project Page Sapiens offers a comprehensive suite for human-centric vision tasks (e.g., 2D pose, part segmentation, depth, normal, etc.). The model family is pretrained on 300 million in-the-wild human images and shows excellent generalization to unconstrained conditions. These models are also designed for extracting high-resolution features, having been natively trained at a 1024 x 1024 image resolution with a 16-pixel patch size. Getting Started Clone the Repository git clone git@github.com:facebookresearch/sapiens.git export SAPIENS_ROOT=/path/to/sapiens Recommended: Lite Installation (Inference-only) For users setting up their own environment primarily for running existing models in inference mode, we recommend the Sapiens-Lite installation. This setup offers optimized inference (4x faster) with minimal dependencies (only PyTorch + numpy + cv2). Full Installation To replicate our complete training setup, run the provided installation script. This will create a new conda environment named sapiens and install all necessary dependencies. cd $SAPIENS_ROOT/_install ./conda.sh Please download the checkpoints from hugging-face. You can be selective about only downloading the checkpoints of interest. Set $SAPIENS_CHECKPOINT_ROOT to be the path to the sapiens_host folder. Checkpoint directory structure: sapiens_host/ +-- detector/ | +-- checkpoints/ | +-- rtmpose/ +-- pretrain/ | +-- checkpoints/ | +-- sapiens_0.3b/ | +-- sapiens_0.6b/ | +-- sapiens_1b/ | +-- sapiens_2b/ +-- pose/ +-- seg/ +-- depth/ +-- normal/ Human-Centric Vision Tasks We finetune sapiens for multiple human-centric vision tasks. Please checkout the list below. * Image Encoder ^[lite] * Pose Estimation ^[lite] * Body Part Segmentation ^[lite] * Depth Estimation ^[lite] * Surface Normal Estimation ^[lite] Easy Steps to Finetuning Sapiens Finetuning our models is super-easy! Here is a detailed training guide for the following tasks. * [Coming Soon] Pose/Seg/Depth * Surface Normal Estimation Acknowledgements & Support & Contributing We would like to acknowledge the work by OpenMMLab which this project benefits from. For any questions or issues, please open an issue in the repository. See contributing and the code of conduct. License This project is licensed under LICENSE. Portions of the project derived from open-source projects are licensed under Apache 2.0. Citation If you use Sapiens in your research, please use the following BibTeX entry. @misc{khirodkar2024_sapiens, title={Sapiens: Foundation for Human Vision Models}, author={Khirodkar, Rawal and Bagautdinov, Timur and Martinez, Julieta and Zhaoen, Su and James, Austin and Selednik, Peter and Anderson, Stuart and Saito, Shunsuke}, year={2024}, eprint={2408.12569}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2408.12569} } About High-resolution models for human tasks. about.meta.com/realitylabs/codecavatars/sapiens/ Resources Readme License View license Code of conduct Code of conduct Security policy Security policy Activity Custom properties Stars 208 stars Watchers 11 watching Forks 13 forks Report repository Releases No releases published Packages 0 No packages published Contributors 2 * @rawalkhirodkar rawalkhirodkar Rawal Khirodkar * @facebook-github-bot facebook-github-bot Facebook Community Bot Languages * Python 89.0% * C++ 6.2% * Cuda 3.7% * Shell 1.1% Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.