https://tensorcookbook.com/

The Tensor Cookbook

What are Tensor Diagrams?

Machine learning involves a lot of tensor manipulation, and it's easy
to lose track of the bigger picture when manipulating
high-dimensional data using notation designed for vectors and
matrices.

It turns out all the trouble with tensors disappears when you instead
represent them using graphs:

Tensor Diagram Example

This book aims to standardize the notation for tensor diagrams by
rewriting the classical "Matrix Cookbook" using this notation.

Tensor diagrams are better than alternative notation like Index
Notation (einsum) because they:

  * Make it easy to spot patterns and symmetries
  * Avoid all trouble with vectorization and Kronecker products
  * Make Matrix Calculus simple and intuitive
  * Represent functions and broadcasting effortlessly

Tensorgrad

is a python library for symbolic tensor manipulation and derivatives
using tensor diagrams. Try it here:

[b, x, y = sp.symbols] Simplify Step by Step

Access Paper:

  * View PDF
  * Github Page

Author:

Thomas Dybdahl Ahle
thomasahle.com and twitter.com/thomasahle.

By Thomas Ahle @ twitter.com/thomasahle.

View PDF Github Page

Topics:

  * Introduction to tensor diagrams
  * Simple and advanced derivatives
  * Statistics and probability
  * Kronecker and Vec operators
  * Special matrices and decompositions
  * Machine learning applications
  * And much more!

How to Cite:

If you use The Tensor Cookbook in your research, please cite it using
the following BibTeX entry:

@book{ahle2024tensorcookbook,
  title={The Tensor Cookbook},
  author={Ahle, Thomas Dybdahl},
  year={2024},
  url={https://tensorcookbook.com},
  note={Version: September, 2024}
}

Other Resources:

  * Penrose Graphical Notation
  * Graphical Tensor Notation for Interpretability
  * The Tensor Product, Demystified
  * Tensor Network Diagrams
  * Matrix Calculus
  * Lectures on Quantum Tensor Networks

---------------------------------------------------------------------

Inspired by The Matrix Cookbook by Kaare Brandt Petersen and Michael
Syskind Pedersen.

(c) 2024 Thomas Dybdahl Ahle. All rights reserved.