Automatic differentiation: forward and reverse mode | Conal Elliott

Cited as the more accessible companion to the paper, and as the most interesting treatment of AD via compiling to categories.

Conal Elliott presents his paper on automatic differentiation, deriving a simple, generalized AD algorithm from a natural specification. Varying the representation of derivatives yields forward and reverse mode algorithms that avoid the complexity and statefulness of backpropagation.