Neural Network Fundamentals
LayerNorm Forward and Backward in NumPy
RMSNorm Forward and Backward Pass
Fold BatchNorm into a Preceding Linear Layer
BatchNorm1d Forward and Backward in NumPy
Xavier and He Weight Initialisation
Fully Connected Linear Layer in NumPy
Residual Block Forward and Backward Pass
Perceptron From Scratch: AND, OR and XOR
Two-Layer NumPy MLP for XOR
ReLU and LeakyReLU Forward and Backward
GELU and SiLU Activations in NumPy
Memory Parameter Accounting
Activation Checkpointing for a Tanh MLP
Autograd Debugging Distributed Training
Forward-Mode Autodiff with Dual Numbers
Scalar Reverse-Mode Autograd Engine
Simulated Data-Parallel MLP Training with All-Reduce
Column- and Row-Parallel Linear Layers