Article
Filter Pruning via Measuring Feature Map Information
Linsong Shao
1,2,3
, Haorui Zuo
2,
*, Jianlin Zhang
2
, Zhiyong Xu
2
, Jinzhen Yao
2
, Zhixing Wang
2
and Hong Li
2
Citation: Shao, L.; Zuo, H.; Zhang, J.;
Xu, Z.; Yao, J.; Wang, Z.; Li, H. Filter
Pruning via Measuring Feature Map
Information. Sensors 2021, 21, 6601.
https://doi.org/10.3390/s21196601
Academic Editor: Alex Alexandridis
Received: 31 July 2021
Accepted: 28 September 2021
Published: 2 October 2021
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
1
Key Laboratory of Optical Engineering, Chinese Academy of Sciences, Chengdu 610200, China;
shaolinsong19@mails.ucas.ac.cn
2
Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu 610200, China;
jlin@ioe.ac.cn (J.Z.); xzy158@163.com (Z.X.); yaojinzhen19@mails.ucas.ac.cn (J.Y.);
E190068@e.ntu.edu.sg (Z.W.); lihong19@mails.ucas.ac.cn (H.L.)
3
University of Chinese Academy of Sciences, Beijing 100049, China
* Correspondence: zuohaorui@sina.com; Tel.: +86-189-8178-8875
Abstract:
Neural network pruning, an important method to reduce the computational complexity of
deep models, can be well applied to devices with limited resources. However, most current methods
focus on some kind of information about the filter itself to prune the network, rarely exploring the
relationship between the feature maps and the filters. In this paper, two novel pruning methods
are proposed. First, a new pruning method is proposed, which reflects the importance of filters by
exploring the information in the feature maps. Based on the premise that the more information there
is, more important the feature map is, the information entropy of feature maps is used to measure
information, which is used to evaluate the importance of each filter in the current layer. Further,
normalization is used to realize cross layer comparison. As a result, based on the method mentioned
above, the network structure is efficiently pruned while its performance is well reserved. Second,
we proposed a parallel pruning method using the combination of our pruning method above and
slimming pruning method which has better results in terms of computational cost. Our methods
perform better in terms of accuracy, parameters, and FLOPs compared to most advanced methods.
On ImageNet, it is achieved 72.02% top1 accuracy for ResNet50 with merely 11.41M parameters
and 1.12B FLOPs.For DenseNet40, it is obtained 94.04% accuracy with only 0.38M parameters and
110.72M FLOPs on CIFAR10, and our parallel pruning method makes the parameters and FLOPs are
just 0.37M and 100.12M, respectively, with little loss of accuracy.
Keywords: model compression; filter pruning; information entropy; normalization
1. Introduction
With the development of deep neural networks in recent years, great success has been
achieved in computer vision applications [
1
–
4
]. However, their apparent effectiveness
is based on increasing storage, memory footprint, computational resources, and energy
consumption, making most advanced Convolutional Neural Networks (CNNs) impossible
to be deployed on edge devices such as cell phones and light devices. Although there are
deep neural network acceleration frameworks such as TensorRT, they cannot reduce the
network model. Therefore, there is still an important demand to reduce the parameters and
floating point operations (FLOPs) of CNNs while keeping the accuracy unchanged. Com-
mon techniques include quantization [
5
–
8
], knowledge distillation [
9
–
11
], and network
pruning [12–16]
. In earlier work, pruning approaches [
17
,
18
] mainly used unstructured
methods to obtain filters for irregular sparsity. To facilitate the deployment of models
on general-purpose hardware and/or the use of basic linear algebra subroutine (BLAS)
libraries, recent works have focused more on structured pruning or filter
pruning [19–21]
,
which simultaneously pursues the reduction of model size and improvement of computa-
tional efficiency.
The existing pruning methods are usually classified into two categories based on
their compact CNN learning process: (1) Pretraining-dependency pruning, which is based
Sensors 2021, 21, 6601. https://doi.org/10.3390/s21196601 https://www.mdpi.com/journal/sensors
