Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
A collection of various deep learning architectures, models, and tips for TensorFlow and PyTorch in Jupyter Notebooks.
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Perceptron | TBD | TBD | |
| Logistic Regression | TBD | TBD | |
| Softmax Regression (Multinomial Logistic Regression) | TBD | TBD | |
| Softmax Regression with MLxtend's plot_decision_regions on Iris | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Multilayer Perceptron | TBD | TBD | |
| Multilayer Perceptron with Dropout | TBD | TBD | |
| Multilayer Perceptron with Batch Normalization | TBD | TBD | |
| Multilayer Perceptron with Backpropagation from Scratch | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Convolutional Neural Network | TBD | TBD | |
| CNN with He Initialization | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Replacing Fully-Connnected by Equivalent Convolutional Layers | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| AlexNet Trained on CIFAR-10 | TBD | TBD | |
| AlexNet with Grouped Convolutions Trained on CIFAR-10 | TBD | TBD |
| Title | Description | Daset | Notebooks |
|---|---|---|---|
| DenseNet-121 Digit Classifier Trained on MNIST | TBD | TBD | |
| DenseNet-121 Image Classifier Trained on CIFAR-10 | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| "All Convolutionl Net" -- A Fully Convolutional Neural Network | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| LeNet-5 on MNIST | TBD | TBD | |
| LeNet-5 on CIFAR-10 | TBD | TBD | |
| LeNet-5 on QuickDraw | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| MobileNet-v2 on Cifar-10 | TBD | TBD | |
| MobileNet-v3 small on Cifar-10 | TBD | TBD | |
| MobileNet-v3 large on Cifar-10 | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Network in Network Trained on CIFAR-10 | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Convolutional Neural Network VGG-16 Trained on CIFAR-10 | TBD | TBD | |
| VGG-16 Smile Classifier Trained on CelebA | TBD | TBD | |
| VGG-16 Dogs vs Cats Classifier | TBD | TBD | |
| Convolutional Neural Network VGG-19 | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| ResNet and Residual Blocks | TBD | TBD | |
| ResNet-18 Digit Classifier Trained on MNIST | TBD | TBD | |
| ResNet-18 Gender Classifier Trained on CelebA | TBD | TBD | |
| ResNet-34 Digit Classifier Trained on MNIST | TBD | TBD | |
| ResNet-34 Object Classifier Trained on QuickDraw | TBD | TBD | |
| ResNet-34 Gender Classifier Trained on CelebA | TBD | TBD | |
| ResNet-50 Digit Classifier Trained on MNIST | TBD | TBD | |
| ResNet-50 Gender Classifier Trained on CelebA | TBD | TBD | |
| ResNet-101 Gender Classifier Trained on CelebA | TBD | TBD | |
| ResNet-101 Trained on CIFAR-10 | TBD | TBD | |
| ResNet-152 Gender Classifier Trained on CelebA | TBD | TBD |
Please note that the following notebooks below provide reference implementations to use the respective methods. They are not performance benchmarks.
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Baseline multilayer perceptron | Cement | A baseline multilayer perceptron for classification trained with the standard cross entropy loss | |
| CORAL multilayer perceptron | Cement | Implementation of Rank Consistent Ordinal Regression for Neural Networks with Application to Age Estimation 2020 | |
| CORN multilayer perceptron | Cement | Implementation of Deep Neural Networks for Rank-Consistent Ordinal Regression Based On Conditional Probabilities 2022 | |
| Binary extension multilayer perceptron | Cement | Implementation of Ordinal Regression with Multiple Output CNN for Age Estimation 2016 | |
| Reformulated squared-error multilayer perceptron | Cement | Implementation of A simple squared-error reformulation for ordinal classification 2016 |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| BatchNorm before and after Activation for Network-in-Network CIFAR-10 Classifier | TBD | TBD | |
| Filter Response Normalization for Network-in-Network CIFAR-10 Classifier | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Siamese Network with Multilayer Perceptrons | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Autoencoder (MNIST) | TBD | TBD | |
| Autoencoder (MNIST) + Scikit-Learn Random Forest Classifier | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Convolutional Autoencoder with Deconvolutions / Transposed Convolutions | TBD | TBD | |
| Convolutional Autoencoder with Deconvolutions and Continuous Jaccard Distance | TBD | TBD | |
| Convolutional Autoencoder with Deconvolutions (without pooling operations) | TBD | TBD | |
| Convolutional Autoencoder with Nearest-neighbor Interpolation | TBD | TBD | |
| Convolutional Autoencoder with Nearest-neighbor Interpolation -- Trained on CelebA | TBD | TBD | |
| Convolutional Autoencoder with Nearest-neighbor Interpolation -- Trained on Quickdraw | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Variational Autoencoder | TBD | TBD | |
| Convolutional Variational Autoencoder | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Conditional Variational Autoencoder (with labels in reconstruction loss) | TBD | TBD | |
| Conditional Variational Autoencoder (without labels in reconstruction loss) | TBD | TBD | |
| Convolutional Conditional Variational Autoencoder (with labels in reconstruction loss) | TBD | TBD | |
| Convolutional Conditional Variational Autoencoder (without labels in reconstruction loss) | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Fully Connected GAN on MNIST | TBD | TBD | |
| Fully Connected Wasserstein GAN on MNIST | TBD | TBD | |
| Convolutional GAN on MNIST | TBD | TBD | |
| Convolutional GAN on MNIST with Label Smoothing | TBD | TBD | |
| Convolutional Wasserstein GAN on MNIST | TBD | TBD | |
| Deep Convolutional GAN (DCGAN) on Cats and Dogs Images | TBD | TBD | |
| Deep Convolutional GAN (DCGAN) on CelebA Face Images | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Most Basic Graph Neural Network with Gaussian Filter on MNIST | TBD | TBD | |
| Basic Graph Neural Network with Edge Prediction on MNIST | TBD | TBD | |
| Basic Graph Neural Network with Spectral Graph Convolution on MNIST | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| A simple single-layer RNN (IMDB) | TBD | TBD | |
| A simple single-layer RNN with packed sequences to ignore padding characters (IMDB) | TBD | TBD | |
| RNN with LSTM cells (IMDB) | TBD | TBD | |
| RNN with LSTM cells (IMDB) and pre-trained GloVe word vectors | TBD | TBD | |
| RNN with LSTM cells and Own Dataset in CSV Format (IMDB) | TBD | TBD | |
| RNN with GRU cells (IMDB) | TBD | TBD | |
| Multilayer bi-directional RNN (IMDB) | TBD | TBD | |
| Bidirectional Multi-layer RNN with LSTM with Own Dataset in CSV Format (AG News) | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| A simple character RNN to generate new text (Charles Dickens) | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Baseline CNN | MNIST | A simple baseline with traditional train/validation/test splits | |
| K-fold with pl_cross | MNIST | A 5-fold cross-validation run using the pl_cross library |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Cyclical Learning Rate | TBD | TBD | |
| Annealing with Increasing the Batch Size (w. CIFAR-10 & AlexNet) | TBD | TBD | |
| Gradient Clipping (w. MLP on MNIST) | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Transfer Learning Example (VGG16 pre-trained on ImageNet for Cifar-10) | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Vanilla Loss Gradient (wrt Inputs) Visualization (Based on a VGG16 Convolutional Neural Network for Kaggle's Cats and Dogs Images) | TBD | TBD | |
| Guided Backpropagation (Based on a VGG16 Convolutional Neural Network for Kaggle's Cats and Dogs Images) | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| MLP in Lightning with TensorBoard -- continue training the last model | TBD | TBD | |
| MLP in Lightning with TensorBoard -- checkpointing best model | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Custom Data Loader Example for PNG Files | TBD | TBD | |
| Using PyTorch Dataset Loading Utilities for Custom Datasets -- CSV files converted to HDF5 | TBD | TBD | |
| Using PyTorch Dataset Loading Utilities for Custom Datasets -- Face Images from CelebA | TBD | TBD | |
| Using PyTorch Dataset Loading Utilities for Custom Datasets -- Drawings from Quickdraw | TBD | TBD | |
| Using PyTorch Dataset Loading Utilities for Custom Datasets -- Drawings from the Street View House Number (SVHN) Dataset | TBD | TBD | |
| Using PyTorch Dataset Loading Utilities for Custom Datasets -- Asian Face Dataset (AFAD) | TBD | TBD | |
| Using PyTorch Dataset Loading Utilities for Custom Datasets -- Dating Historical Color Images | TBD | TBD | |
| Using PyTorch Dataset Loading Utilities for Custom Datasets -- Fashion MNIST | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Generating Validation Set Splits | TBD | TBD | |
| Dataloading with Pinned Memory | TBD | TBD | |
| Standardizing Images | TBD | TBD | |
| Image Transformation Examples | TBD | TBD | |
| Char-RNN with Own Text File | TBD | TBD | |
| Sentiment Classification RNN with Own CSV File | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Gradient Checkpointing Demo (Network-in-Network trained on CIFAR-10) | TBD | TBD |
| Title | Description | Notebooks |
|---|---|---|
| Using Multiple GPUs with DataParallel -- VGG-16 Gender Classifier on CelebA | TBD | |
| Distribute a Model Across Multiple GPUs with Pipeline Parallelism (VGG-16 Example) | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| PyTorch with and without Deterministic Behavior -- Runtime Benchmark | TBD | TBD | |
| Sequential API and hooks | TBD | TBD | |
| Weight Sharing Within a Layer | TBD | TBD | |
| Plotting Live Training Performance in Jupyter Notebooks with just Matplotlib | TBD | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Getting Gradients of an Intermediate Variable in PyTorch | TBD | TBD |
| Title | Description | Notebooks |
|---|---|---|
| Chunking an Image Dataset for Minibatch Training using NumPy NPZ Archives | TBD | |
| Storing an Image Dataset for Minibatch Training using HDF5 | TBD | |
| Using Input Pipelines to Read Data from TFRecords Files | TBD | |
| Using Queue Runners to Feed Images Directly from Disk | TBD | |
| Using TensorFlow's Dataset API | TBD |
| Title | Dataset | Description | Notebooks |
|---|---|---|---|
| Saving and Loading Trained Models -- from TensorFlow Checkpoint Files and NumPy NPZ Archives | TBD | TBD |
| Title | Description | Notebooks |
|---|---|---|
| TorchMetrics | How do we use it, and what's the difference between .update() and .forward()? |