https://github.com/apple/corenet Skip to content Navigation Menu Toggle navigation Sign in * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} apple / corenet Public * Notifications * Fork 28 * Star 1.1k * CoreNet: A library for training deep neural networks License View license 1.1k stars 28 forks Branches Tags Activity Star Notifications * Code * Issues 1 * Pull requests 4 * Security * Insights Additional navigation options * Code * Issues * Pull requests * Security * Insights apple/corenet This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main BranchesTags Go to file Code Folders and files Last Last Name Name commit commit message date Latest commit History 3 Commits assets assets corenet corenet mlx_examples mlx_examples projects projects tests tests tools tools tutorials tutorials .dockerignore .dockerignore .flake8 .flake8 .gitattributes .gitattributes .gitignore .gitignore ACKNOWLEDGEMENTS ACKNOWLEDGEMENTS CODE_OF_CONDUCT.md CODE_OF_CONDUCT.md CONTRIBUTING.md CONTRIBUTING.md LICENSE LICENSE Makefile Makefile README.md README.md conftest.py conftest.py pyproject.toml pyproject.toml requirements-optional.txt requirements-optional.txt requirements.txt requirements.txt setup.py setup.py tox.ini tox.ini View all files Repository files navigation * README * Code of conduct * License CoreNet: A library for training deep neural networks CoreNet is a deep neural network toolkit that allows researchers and engineers to train standard and novel small and large-scale models for variety of tasks, including foundation models (e.g., CLIP and LLM), object classification, object detection, and semantic segmentation. Table of contents * What's new? * Research efforts at Apple using CoreNet * Installation * Directory Structure * Maintainers * Contributing to CoreNet * License * Relationship with CVNets * Citation What's new? * April 2024: Version 0.1.0 of the CoreNet library includes + OpenELM + CatLIP + MLX examples Research efforts at Apple using CoreNet Below is the list of publications from Apple that uses CoreNet: * OpenELM: An Efficient Language Model Family with Open-source Training and Inference Framework * CatLIP: CLIP-level Visual Recognition Accuracy with 2.7x Faster Pre-training on Web-scale Image-Text Data * Reinforce Data, Multiply Impact: Improved Model Accuracy and Robustness with Dataset Reinforcement * CLIP meets Model Zoo Experts: Pseudo-Supervision for Visual Enhancement * FastVit: A Fast Hybrid Vision Transformer using Structural Reparameterization * Bytes Are All You Need: Transformers Operating Directly on File Bytes * MobileOne: An Improved One millisecond Mobile Backbone * RangeAugment: Efficient Online Augmentation with Range Learning * Separable Self-attention for Mobile Vision Transformers (MobileViTv2) * CVNets: High performance library for Computer Vision, ACM MM'22 * MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer, ICLR'22 Installation You will need Git LFS (instructions below) to run tests and Jupyter notebooks (instructions) in this repository, and to contribute to it so we recommend that you install and activate it first. On Linux we recommend to use Python 3.10+ and PyTorch (version >= v2.1.0), on macOS system Python 3.9+ should be sufficient. Note that the optional dependencies listed below are required if you'd like to make contributions and/or run tests. For Linux (substitute apt for your package manager): sudo apt install git-lfs git clone git@github.com:apple/corenet.git cd corenet git lfs install git lfs pull # The following venv command is optional, but recommended. Alternatively, you can create and activate a conda environment. python3 -m venv venv && source venv/bin/activate python3 -m pip install --editable . To install optional dependencies for audio and video processing: sudo apt install libsox-dev ffmpeg For macOS, assuming you use Homebrew: brew install git-lfs git clone git@github.com:apple/corenet.git cd corenet cd \$(pwd -P) # See the note below. git lfs install git lfs pull # The following venv command is optional, but recommended. Alternatively, you can create and activate a conda environment. python3 -m venv venv && source venv/bin/activate python3 -m pip install --editable . To install optional dependencies for audio and video processing: brew install sox ffmpeg Note that on macOS the file system is case insensitive, and case sensitivity can cause issues with Git. You should access the repository on disk as if the path were case sensitive, i.e. with the same capitalization as you see when you list the directories ls. You can switch to such a path with the cd $(pwd -P) command. Directory Structure This section provides quick access and a brief description for important CoreNet directories. Description Quick Access Getting Started +-- tutorials +-- train_a_new_model_on_a_new_dataset_from_scratch.ipynb +-- guide_slurm_and_multi_node_training.md Working with the +-- clip.ipynb examples is an easy way +-- semantic_segmentation.ipynb to get started with +-- object_detection.ipynb CoreNet. Training Recipes CoreNet provides reproducible training recipes, in addition to +-- projects the pretrained model +-- byteformer weights and checkpoints +-- catlip (*) for the publications +-- clip that are listed in +-- fastvit projects/ directory. +-- mobilenet_v1 +-- mobilenet_v2 Publication project +-- mobilenet_v3 directories generally +-- mobileone contain the following +-- mobilevit contents: +-- mobilevit_v2 +-- openelm (*) * README.md provides +-- range_augment documentation, links +-- resnet to the pretrained +-- vit weights, and citations. * / (*) Newly released. .yaml provides configuration for reproducing the trainings and evaluations. MLX Examples MLX examples demonstrate +--mlx_example how to run CoreNet +-- clip models efficiently on +-- open_elm Apple Silicon. Please find further information in the README.md file within the corresponding example directory. Model Implementations Models are organized by tasks (e.g. "classification"). You can find all model implementations for each +-- corenet task in the +-- modeling corresponding task +-- models folder. +-- audio_classification +-- classification Each model class is +-- detection decorated by a +-- language_modeling @MODEL_REGISTRY.register +-- multi_modal_img_text (name="", +-- segmentation type="") decorator. To use a model class in CoreNet training or evaluation, assign moels. .name = in the YAML configuration. +-- corenet +-- data Datasets +-- datasets +-- audio_classification +-- classification Similarly to the models, +-- detection datasets are also +-- language_modeling categorized by tasks. +-- multi_modal_img_text +-- segmentation +-- corenet +-- loss_fn +-- metrics Other key directories +-- optims | +-- scheduler +-- train_eval_pipelines In this section, we have +-- data highlighted the rest of | +-- collate_fns the key directories that | +-- sampler implement classes | +-- text_tokenizer corresponding to the | +-- transforms names that are | +-- video_reader referenced in the YAML +-- modeling configurations. +-- layers +-- modules +-- neural_augmentor +-- text_encoders Maintainers This code is developed by Sachin, and is now maintained by Sachin, Maxwell Horton, Mohammad Sekhavat, and Yanzi Jin. Previous Maintainers * Farzad Contributing to CoreNet We welcome PRs from the community! You can find information about contributing to CoreNet in our contributing document. Please remember to follow our Code of Conduct. License For license details, see LICENSE. Relationship with CVNets CoreNet evolved from CVNets, to encompass a broader range of applications beyond computer vision. Its expansion facilitated the training of foundational models, including LLMs. Citation If you find our work useful, please cite the following paper: @inproceedings{mehta2022cvnets, author = {Mehta, Sachin and Abdolhosseini, Farzad and Rastegari, Mohammad}, title = {CVNets: High Performance Library for Computer Vision}, year = {2022}, booktitle = {Proceedings of the 30th ACM International Conference on Multimedia}, series = {MM '22} } About CoreNet: A library for training deep neural networks Resources Readme License View license Code of conduct Code of conduct Activity Custom properties Stars 1.1k stars Watchers 15 watching Forks 28 forks Report repository Releases No releases published Packages 0 No packages published Languages * Python 99.7% * Other 0.3% Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.