https://nn.labml.ai

home

Github Join Slact Twitter

#

LabML Neural Networks

This is a collection of simple PyTorch implementations of neural
networks and related algorithms. These implementations are documented
with explanations, and the website renders these as side-by-side
formatted notes. We believe these would help you understand these
algorithms better.

We are actively maintaining this repo and adding new implementations.

Modules

 Transformers

Transformers module contains implementations for multi-headed
attention and relative multi-headed attention.

  * GPT Architecture
  * GLU Variants
  * kNN-LM: Generalization through Memorization
  * Feedback Transformer
  * Switch Transformer

 Recurrent Highway Networks

 LSTM

 HyperNetworks - HyperLSTM

 Capsule Networks

 Generative Adversarial Networks

  * GAN with a multi-layer perceptron
  * GAN with deep convolutional network
  * Cycle GAN

 Sketch RNN

 Reinforcement Learning

  * Proximal Policy Optimization with Generalized Advantage
    Estimation
  * Deep Q Networks with with Dueling Network, Prioritized Replay and
    Double Q Network.

 Optimizers

  * Adam
  * AMSGrad
  * Adam Optimizer with warmup
  * Noam Optimizer
  * Rectified Adam Optimizer
  * AdaBelief Optimizer

Installation

pip install labml_nn

Citing LabML

If you use LabML for academic research, please cite the library using
the following BibTeX entry.

@misc{labml,
 author = {Varuna Jayasiri, Nipun Wijerathne},
 title = {LabML: A library to organize machine learning experiments},
 year = {2020},
 url = {https://lab-ml.com/},
}