http://machinethink.net/blog/compressing-deep-neural-nets/

------- before 12 Aug 2018 -------

• SqueezeNet, see tutorial_squeezenet.py
• MobileNet, see tutorial_mobilenet.py
• BinaryNet, see mnist cifar10.
• Ternary Weight Network, see mnist cifar10.
• DoReFa-Net, see mnist cifar10.

@XJTUWYD add two compress strategies tenary weight network and dorefa-net into tensorlayer, did two experiments to compare the accuracy of different compress strategies based on mnist and cifar-10.
the result of the experiment is below:

@XJTUWYD : bnn is a excellent work in the compression of neuron network but it can not get a satisfied accuracy on relative large datasets, in order to solve the problem, tenary weight networks and dorefa were put forward. I add 4 apis for tensorlayer, Tenary Denselayer, TenaryConv2d, DorefaDenselayer, and DorefaConv2d . I perform 6 experiment based on mnist and cifar10，the details are in thr tutorials. Finally thank you very much for the help of HaoDong, LuoMai, and Igarithm.

------- before 15 March 2018 -------

Hi, I am trying to make TensorLayer/ TensorFlow to support BinaryNet, Xnor-Net, SqueezeNet, MobileNet, ShuffleNet, DoReFaNet, Channel Pruning and etc. Feel free to discuss and add more information here. ~

Paper List

• A Survey of Model Compression and Acceleration for Deep Neural Networks (end of 2017)
• More in arXiv-Sanity

1. Quantization

• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations paper
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1 paper itayhubara/BinaryNet.tf
• Ternary Weight Networks paper
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks paper AngusG/tensorflow-xnor-bnn
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients paper
• Towards Accurate Binary Convolutional Neural Network paper

2. Pruning

• Channel Pruning for Accelerating Very Deep Neural Networks paper

3. Structure

• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size paper TensorLayer example
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications paper
• ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices paper code1 code2
• 纵览轻量化卷积神经网络：SqueezeNet、MobileNet、ShuffleNet、Xception
• CNN 模型压缩与加速算法综述

About TensorLayer & TensorFlow

• tf.custom_gradient
• Quantized Layers

Others

• How to Quantize Neural Networks with TensorFlow
• how-to-quantize-neural-networks-with-tensorflow 2016 May
• Embedded Vision Summit
• Deep Compression and EIE
• EIE: Efficient Inference Engine on Compressed Deep Neural Network